615 results about "Backdoor" patented technology

The invention discloses a federated learning backdoor defense method based on pruning and fine tuning in the technical field of artificial intelligence and network security, the method realizes defense through two core mechanisms of dynamic pruning and gradient constraint fine tuning, and the method comprises the following steps: firstly, calculating a sensitivity score based on a neuron activation frequency and a weight outlier degree; dynamically identifying and cutting redundant neurons utilized by a backdoor, and blocking an abnormal activation path; secondly, gradient direction consistency detection and amplitude constraint are introduced in the fine tuning stage, and a malicious client is inhibited from reconstructing a back door through an abnormal gradient; the server continuously purifies model parameters and enhances robustness by cyclically executing pruning, fine tuning and aggregation operations; the method does not need to depend on an extra clean data set, strictly follows a federated learning privacy protection principle, reduces communication overhead through lightweight pruning, maintains main task performance in combination with gradient constraint, is suitable for a federated learning scene in which edge equipment participates, and effectively balances a defense effect and model stability.

Disclosed in the present invention is a federated learning method against a backdoor attack. Firstly, a few benign model updates having a relatively high confidence are selected by means of clustering-based voting, then the selected models are considered as benign modes of model updates, and on the basis of an anomaly detection method of a variational auto-encoder, more benign model updates are progressively selected from among the remaining candidate model updates, so that the scale of a group of the selected benign model updates is continuously expanded. The present invention does not depend on differential privacy, weight pruning, and learning rate adjustment and merely sightly changes a protocol of original federated learning, and therefore, on one hand, the impact on the accuracy of a global model is relatively small, and on the other hand, integration with existing federated learning systems is facilitated. Compared with existing federated learning methods against backdoor attacks, the present invention has relatively high adaptability.

The invention provides a privacy protection federated distillation and backdoor defense method for large model fine tuning, and belongs to the technical field of artificial intelligence security and federated learning, and the method comprises the steps: 1, carrying out the distillation and core representation extraction of a data set based on contribution degree weighted federated pre-training and local neural feature function matching; step 2, self-adaptive noise back door defense processing based on multi-feature fusion; according to the method, a dataset distillation mechanism based on neural feature function matching and a self-adaptive noise defense strategy are adopted, so that effective balance of the large model among data simplification, privacy protection and backdoor defense robustness is realized; the method is of great significance in improving the safety and reliability of an artificial intelligence system in a distributed environment.

The invention discloses a backdoor attack chain method of a code-driven body agent based on a large language model. The method comprises the following steps: creating an initial poisoning prompt by using a generative adversarial network; the poisoning prompt is optimized through iterative confrontation training between the generator and the discriminator; inputting the optimized poisoning prompt into the target large language model; deploying the program with the backdoor defect to the intelligent agent with the body; when the body agent runs a program with a backdoor defect, a visual trigger in the environment activates the backdoor, causing the body agent to perform a preset specific behavior. By using poisoning prompts optimized by the generative adversarial network, the present invention can generate backdoor codes that are difficult to detect, thus having significant advantages in concealment and effectiveness.

The invention discloses a continuous verification method for the forgotten right of a federal reduction learning client, and the method comprises the steps: embedding an EndoraMark mark in the data of a forgotten client, tracking the performance of a global model in a plurality of training rounds, and guaranteeing that whether the data of the federal reduction learning client is successfully forgotten or not can be accurately verified. According to the method, the forgetting effect is evaluated by adopting the backdoor attack, and the potential privacy leakage problem is avoided. According to the method, an improved scaling strategy and an imprint enhancement data enhancement method are introduced, the durability of the private imprint in the federated learning model is enhanced, the influence of model benign updating on imprint dilution is reduced, and verification of the forgetting right of the client side still has high effectiveness in long-term model training.

Technology described herein relates to managing backdoor vulnerabilities of a computer system. A system for the managing can comprise a processor, and a memory that stores executable instructions that, when executed by the processor, facilitate performance of operations, comprising analyzing system information relating to operation of an application programming interface (API); based on a result of the analyzing of the system information, constructing a call function for execution of the API and executing the call function; based on monitoring a data flow of the system with respect to the execution of the API, generating impact data representative of an impact of the execution of the API; and determining whether the impact is counter to historical functioning of the system as represented by historical functioning data.

The invention relates to the field of software security, and particularly discloses a software supply chain security analysis method which comprises the following steps: S1, collecting data of components of a software supply chain, and calculating a historical security risk weight for each component in the supply chain; s2, collecting behavior data of the software during operation, and performing association analysis in combination with the supply chain data; s3, preprocessing the collected behavior data and supply chain data, and extracting key features; according to the software supply chain security analysis method, through dynamic behavior analysis, runtime threats, such as zero-day vulnerabilities and hidden backdoor threats, which cannot be detected by static analysis can be found, the coverage rate of threat detection is remarkably improved, meanwhile, historical security risk weights are introduced, historical security problems of supply chain components are quantified, and the security of the software supply chain is improved. And intelligent risk assessment and threat traceability are realized by combining a knowledge graph and a graph neural network technology.

The invention provides a federated learning backdoor defense method based on singular value decomposition and model weight amplification, and the method comprises the steps: carrying out the normalization processing of a model updating parameter locally trained by a client, and obtaining a normalized model updating parameter; on the basis of the normalized model updating parameters, model updating parameters after dimension reduction are obtained; performing clustering algorithm processing on the model updating parameters after dimension reduction to obtain clustered clusters; obtaining cluster model parameters based on the clustered clusters; combining the cluster model parameters into a cluster model parameter matrix; performing singular value decomposition on the cluster model parameter matrix to obtain a singular vector; obtaining a trust score through the singular vector; obtaining global model update parameters based on the trust score; calculating by utilizing the global model updating parameters to obtain a global model; and adding Gaussian noise to the global model through a differential privacy mechanism to obtain a final global model. According to the method, backdoor attacks can still be effectively resisted under the scene that the client data sets are non-independent and identically distributed.

The invention relates to the field of personal intelligent security evaluation, and particularly discloses a chain reasoning hidden backdoor vulnerability detection method of a vision-language-action model, which comprises the following steps of: respectively injecting micro pixel disturbance and rare character marks into vision and language input; on the basis of model autoregression prediction characteristics, designing a hybrid reasoning chain fusing normal reasoning steps and abnormal backdoor branches; adopting prefix tuning to take the hybrid reasoning sequence as a pluggable prefix injection model; and generating the vulnerability sensitivity of the abnormal action instruction through the systematic verification process detection model. Compared with an existing method, the method has the advantages that a nondestructive testing mechanism based on prefix adjustment and optimization does not need to modify model parameters or depend on training data, and the safety and reproducibility of detection are guaranteed; a multi-mode triggering mechanism is constructed, and the hidden vulnerability of the model in a complex scene is effectively revealed; the abnormal branches and the normal process are fused in a chain mode, and the defense capability of the model for the concealment logic offset can be systematically evaluated.

The invention discloses a back door defense method based on forgetting learning and implicit hypergradient. The method comprises the steps of assuming a back door label, generating a possible back door trigger corresponding to the label and performing parameter search, obtaining a static back door trigger, adding disturbance, enabling an optimizer to optimize the back door label in a direction deviating from a correct label, and adapting to a dynamic back door trigger. Carrying out probability superposition by using the balanced sample; according to the parameters of the static trigger and the dynamic trigger and the prediction probability accumulative distribution, calculating a comprehensive score of the backdoor tag, and calculating a target tag; information such as model features is extracted as a training set, and the trained backdoor detection model is used for model backdoor detection; constructing a mixed data set, and performing data enhancement on the mixed data set; and performing forgetting training and multi-stage implicit super-gradient training on the classification model by using the enhanced data set, and deleting the back door of the classification model. According to the method, the backdoor is effectively deleted from the neural network model, so that the safety and reliability of the model are ensured.

The invention relates to the technical field of chip verification, in particular to a simulation verification system based on a DFI interface, the system comprises a memory controller, a verification component and a memory model, the memory controller is connected with the verification component through the DFI interface, the memory controller is used for transmitting an operation command and first data through the DFI interface, the verification component is connected with the memory model, and the memory model is connected with the verification component. The verification component is used for checking the operation command and the first data, analyzing the operation command to obtain address information and calling the memory model to operate the first data, and the memory model is used for operating the first data according to the received operation command, address information and first data sent by the verification component. It can be known that the verification component and the memory model can be constructed based on the verification language compatible with the verification platform, so that initialization only needs to configure the memory controller through the bus, the initialization time of the simulation verification system is saved, a more flexible backdoor data loading mode is provided, and the simulation verification efficiency is improved.

The invention provides a parameter negotiation method for a high-speed serial interface, which is applied to an adaptation layer of any chip in a chip pair for data interaction based on UCIE, and comprises the following steps: requesting a home terminal physical layer to enter an active state under the condition that the home terminal physical layer completes link training; under the condition that a home terminal physical layer enters an active state and a backdoor selection bit is not set, determining a home terminal performance index based on a link training result and home terminal performance parameters pre-configured by a user; determining a target protocol stack to be subjected to parameter negotiation and a to-be-negotiated protocol type of the target protocol stack based on the home terminal performance index, and performing parameter negotiation handshake with the opposite terminal based on the to-be-negotiated protocol type of the target protocol stack; under the condition that the parameter negotiation is successful, the target protocol and the target data format are determined based on the result of the parameter negotiation handshake, and the indication information of the target protocol and the target data format is sent to a home terminal protocol layer, so that the parameter negotiation efficiency of the high-speed serial interface can be improved.

Systems, computer program products, and methods are described herein for detecting manipulation of training data used for machine learning models. The present disclosure is configured to receive an interaction originating from an end-point device; validate the interaction via a primary model; identify a set of triggers within a backdoor model, where the backdoor model is modeled off the primary model capable of undergoing stress testing associated with a set of triggers; pause the interaction upon identification of the set of triggers within the backdoor model; transmit the identified set of triggers to an autonomous artificial general intelligence (AGI) agent, generate a set of synthetic data via the autonomous AGI agent, where the set of synthetic data removes the set of triggers from the primary model; and distribute the set of synthetic data to the primary model to correct the set of triggers within the primary model.

The invention discloses a robust backdoor defense method and system without auxiliary data dependence, and belongs to the technical field of deep learning model security and artificial intelligence defense. The internal relation between back door injection and sample pollution is analyzed through confrontation disturbance, and a clean sample and a polluted sample are preliminarily isolated from a toxic data set; performing reverse learning and re-learning on the selected clean sample, and training to obtain an enhanced poisoning model; polluted samples and clean samples are accurately identified by enhancing a poisoning model, model training is performed based on the clean samples, a back-door-free robust model is finally obtained, finally, the polluted samples are re-labeled and are used for fine adjustment of the back-door-free robust model, and the robustness of the model is further improved. The method does not depend on any auxiliary clean data set, various backdoor attacks can be effectively resisted, and meanwhile the high precision of the model is kept. According to the method, the vacancy of robust backdoor defense model training under the condition of no auxiliary data in the prior art is filled.

The invention discloses a frequency domain stealth backdoor attack method based on a generative adversarial network, and belongs to the technical field of image data processing, and the method comprises the steps: obtaining a clean sample set D1 and a classification model; constructing a generative adversarial network, and constructing a loss function based on picture similarity loss, frequency domain consistency loss and adversarial loss; training the generative adversarial network by using the D1 to obtain a generative model; constructing a clean data set and a backdoor data set based on the D1, a target label t of the backdoor attack and the generative model; constructing multi-layer MMD loss; and constructing total loss based on MMD and MSE, and performing poison training on the classification model to obtain a backdoor model. According to the method, a multi-domain disturbance generation network is constructed, pixel-level disturbance is generated in a spatial domain, discrete wavelet transform is introduced into a frequency domain to constrain frequency domain characteristics of back door disturbance, poisoning samples have concealment in the spatial domain and the frequency domain, multi-layer MMD loss is introduced, the deep characteristic distribution difference between clean samples and back door samples is reduced, and high-concealment attack is achieved.

The invention discloses a federated learning backdoor attack defense method based on layer perception detection, which relates to the technical field of artificial intelligence security, and comprises the steps of client training and uploading, malicious client identification, backdoor key layer detection and deletion, robust aggregation updating and adversarial training for enhancing robustness. According to the method, the model parameters uploaded by the clients are subjected to clustering analysis, and the maximum benign cluster is identified by adopting an unsupervised clustering method, so that instability caused by single threshold judgment is avoided, and the benign client and the malicious client can be distinguished; and meanwhile, layer perception detection and elimination are used in the scheme, so that the backdoor introduced by a malicious client can be effectively identified and eliminated on the premise of ensuring the global model precision, and the influence of backdoor attack on the model is inhibited.

A fraud detection apparatus comprises: a function extraction part that refers to a target function list showing functions to be analyzed and analyzes a program to extract a target function; a structure extraction part that analyzes the program to extract an execution path and a conditional branch; a conditional branch scoring part that refers to a score list showing the probability of meeting the condition of a conditional branch and assigns a score to each of the extracted conditional branches to create a conditional branch score table; a reachability probability calculation part that calculates the probability of reaching the target function from the scores for conditional branches included in the execution path on the basis of the conditional branch score table; and a backdoor determination part that reports an execution path having a low reachability probability as a path with a high probability of being a backdoor execution path.

The invention provides a federal learning backdoor attack defense method based on adaptive clustering, and aims to solve the problems that the defense capability is insufficient and backdoor attacks with the malicious client proportion exceeding 50% cannot be effectively defended under the condition of data non-independent identical distribution. An existing federal learning backdoor attack defense method based on clustering generally adopts a diclustering strategy, and has the problems that the malicious client identification accuracy is not high, and defense fails to the malicious client in a proportion of more than 50%. In order to solve the problems, a Louvain community discovery algorithm is applied to perform adaptive clustering on model parameters, and a historical cumulative credibility updating mechanism and an abnormal value cutting strategy are introduced, so that malicious clients are accurately identified and effective defense is realized on the premise of ensuring the main task precision of a global model.

The present invention discloses a deep learning backdoor attack method and device based on an ordinal network and a medium, which belongs to the technical field of neural network security. The method comprises: obtaining a training sample image; generating an ordinal network based on the training sample image, wherein the ordinal network is used for representing an exact structure of an image; and generating a trigger by using the ordinal network. Through the above method, and the device and the medium for realizing the above method, the present invention uses the ordinal network to generate the trigger, improves the concealability of a poisoning sample compared with poisoning samples generated by other attack methods, and can promote the further research of a hidden backdoor attack defense method in the academic circle.

The invention discloses a federal self-supervised learning backdoor attack method and system based on illusion positive example entanglement, and the method comprises the steps: setting a target of an attacker to embed a backdoor into an image encoder during a pre-training period, and guaranteeing that a downstream classifier trained on the image encoder predicts an image with a trigger as a target class; the limited poisoning data is fully utilized, and the feature similarity of the same trigger sample is improved through the illusion enhancement module, so that the malicious client can effectively embed the backdoor into the global encoder. In the training process, a forged positive sample is generated, learning of the model in a feature space is misguided intentionally, poisoning data is utilized to the maximum extent, an entanglement loss function is designed to optimize a target, the attack efficiency is kept in the whole model aggregation process, and the attenuation risk is reduced. According to the invention, a new perspective and important challenge can be provided for the security research of the federated learning system, and the improvement and development of defense capability are promoted.

The invention belongs to the field of agent security, and discloses a multi-agent application program attack assessment method and a related device, and the method comprises the steps: firstly, constructing a poisoning attack prompt word and a backdoor trigger through obtaining an attack motivator and an attack constraint, and combining to generate a backdoor attack sample; then identifying a potential easily-poisoned agent in the multi-agent application program, and inserting a backdoor attack sample into a specified site to generate a poisoned agent; carrying out multiple rounds of agent communication in the multi-agent application program based on the poisoning agent, and simulating an interaction process in a real attack scene to obtain a poisoning multi-agent application program; and finally, a statement with a backdoor trigger is input, and an attack evaluation report is generated according to an output result of the poisoning multi-agent application program, so that the comprehensiveness of poisoning and backdoor attack robustness evaluation of the multi-agent application program is improved, and a powerful basis is provided for optimization of the multi-agent application program. And the agents with excellent performance in the multi-agent application program can be mined.

The invention belongs to the technical field of artificial intelligence security, and relates to a defense method for model backdoor attacks. The invention aims to solve the problem that the accuracy of the model is remarkably reduced due to the existing defense technology. The method combines a backdoor label detection method and a model fine tuning method, and is realized by the following steps of: firstly, detecting a backdoor label by utilizing a nerve cleaning method, and identifying a potential backdoor trigger; secondly, generating an adversarial sample by using a projection gradient descent algorithm; then, carrying out first-stage adversarial fine tuning training to reduce the sensitivity of the model to a backdoor trigger; and finally, performing second-stage fine tuning training through the clean sample, and gradually recovering the classification performance of the model on the original data.

The invention discloses a rumor detector-oriented backdoor attack generation method, system and device, and relates to the technical field of rumor detection and backdoor attack.The method comprises the steps of obtaining a rumor propagation graph; respectively generating importance scores of node structures and features of the propagation graph by adopting degree centrality and similarity between nodes and root nodes, and screening out attached nodes according to the importance scores; constructing a self-adaptive feature generator, and generating an aggressive self-adaptive trigger node by taking the node feature of the attached node as input; meanwhile, similarity constraints are added to the node features, so that the trigger node features can be adaptively close to the features of the attached nodes; a self-adaptive feature generator is used for self-adaptively generating a corresponding node feature for each trigger node, the trigger nodes with the node features in a certain proportion are injected into the attached nodes, and a back door graph is generated and used for attacking the rumor detector; according to the method, the performance of a propagation-based rumor detector is effectively destroyed, and the method can be transferred among different architectures.

The invention provides a federal learning backdoor attack defense method based on a multi-layer cooperative defense strategy, and belongs to the technical field of network data security. Aiming at the defects in the prior art that only single-point protection at a certain stage is covered in federated learning, systematic robust defense for a whole process is lacked, backdoor attack detection capability is weak, model recovery efficiency is low and the like, the invention provides a multi-layer defense framework penetrating through a whole period (training, aggregation and deployment) of federated learning. A special information flow channel, a global security evaluation system and a self-adaptive resource allocation strategy are established among the layers, and the three defense layers are organically combined into a cooperative combat whole. The cooperation mechanism not only improves the efficiency of a single defense layer, but also realizes' 1 + 1 + 1gt '; and 3 ''through the synergistic effect, an omnibearing and intelligent backdoor attack defense solution is provided for a federated learning system.

Various embodiments provide a robust backdoor attack on machine learning (ML)-based detection systems that can be applied to demonstrate and identify vulnerabilities thereof. In various embodiments, an artificial intelligence (AI)-based Trojan attack is generated and implanted inside a ML model trained for classification and / or detection tasks, and the AI-based Trojan attack can be triggered by specific inputs to manipulate the expected outputs of the ML model. Analysis of the behavior of an ML model having the AI-based Trojan implanted (and / or triggered) then enables identification of vulnerabilities of the ML model and further enables the design of ML models with improved security. Various embodiments of the present disclosure provide a fast and cost-effective solution in achieving 100% attack success rate that significantly outperforms adversarial attacks on ML models, thereby improving applicability and depth in testing ML-based detection systems.

The invention relates to the technical field of network security, in particular to a defense method and system for federated learning backdoor attacks, a server initializes a federated learning global model, and identifies at least one key layer easy to implant a backdoor in the global model; distributing the current global model parameters to a plurality of clients, receiving model update information uploaded by each client, extracting a multi-dimensional gradient feature vector of each client based on gradient information of each client in a key layer, calculating a mahalanobis distance anomaly score of each client relative to the gradient distribution of the whole client, and calculating the mahalanobis distance anomaly score of each client according to the mahalanobis distance anomaly score; and screening and rejecting the clients which are judged to be abnormal, aggregating model updates uploaded by the clients which are judged to be benign, and generating a new global model. Malicious updating is effectively isolated while the performance of the main task is ensured, and the robustness and safety of a global model are improved.

The invention discloses a text style backdoor defense method based on multi-granularity variant generation and style immunization. The method comprises the following steps: capturing a text style and content by combining explicit and implicit features; generating high-quality text variants on a plurality of granularities such as lexical, syntactic, style, context and the like by utilizing a large language model; carrying out label correction on the suspicious samples based on multi-dimensional risk assessment, and carrying out voting decision making by utilizing style neutralization variants and the like; and finally, through style invariant representation learning, style separation and style contrast training, the robustness of the model to style change is improved. According to the method, an explicit detection trigger is not needed, various attacks including style backdoors can be effectively defended, and through a systematic framework and a self-adaptive strategy, the safety and robustness of the model are remarkably improved while the normal performance of the model is ensured.

The invention discloses a backdoor attack detection method, device and equipment, a medium and a program product. The method comprises the following steps: acquiring a benign sample set and input data; transplanting the local features of the input data to each benign sample in the benign sample set to obtain a disturbance sample set; determining a prediction category of the disturbance sample set and a prediction category of the benign sample set based on the target classification model; and according to the prediction category of the benign sample set and the prediction category of the disturbance sample set, determining whether the input data is backdoor data. Local features of input data are transplanted to a benign sample to obtain a disturbance sample set by utilizing the strong correlation characteristic of model implanted backdoor attack and disturbance styles, backdoor attack detection is performed on the consistency of prediction categories of the benign sample set and the disturbance sample set based on a target classification model, the universality of a backdoor attack mode is high, and the accuracy of backdoor attack detection is high. The method is not constrained by an attack mode, and has relatively high detection accuracy for back door attacks irrelevant to input and back door attacks relevant to input.

The invention discloses a graph backdoor encoder defense method and device, and a readable storage medium. The method comprises the steps that at least two label-free enhanced data sets are generated based on a preset training data set, a to-be-processed encoder is trained based on the enhanced data sets to obtain a teacher encoder, and the training data set is a downstream data set or a subset of the downstream data set; defining identical attention operators in corresponding layers of the teacher encoder and the to-be-processed encoder, wherein the attention operators are used for outputting layer attention maps of corresponding encoder layers; calculating the distribution offset of the layer attention map of the teacher encoder and the layer attention map of the encoder to be processed in the same encoder layer; and fixing the parameters of the teacher encoder, and updating the parameters of the to-be-processed encoder based on the distribution offset. Compared with the prior art, through lightweight deployment, on the premise that the model precision is not reduced, the attack success rate of backdoor attack on graph self-supervised learning is effectively reduced, and the backdoor defense capability of the system is improved.

The invention provides a 3D point cloud defense method, device and equipment based on CLIP guidance, and a medium, relates to the technical field of point cloud defense, and aims to enhance the robustness and security of 3D point cloud data in the face of adversarial attack by using a contrast language-image pre-training model. According to the method, a new defense framework is provided by combining geometric and semantic features, triggers in backdoor attacks can be effectively identified and inhibited, meanwhile, the geometric structure of point cloud data is repaired, and the classification performance is improved. Specifically, the method includes projecting a 3D point cloud into a multi-view 2D depth map, extracting cross-modal features using a CLIP model, and performing semantically guided 3D reconstruction through a variational auto-encoder (VAE) to generate a repaired point cloud that retains key information and neutralizes perturbations. The method shows excellent performance in various complex scenes, and provides powerful support for the safety of the 3D vision technology in key application.