278 results about "Injection attacks" patented technology

Systems and methods for implementing prevention of prompt injection attacks on large language models by tokenization of structured data elements is presented. The systems and methods replace one or more data elements in a database response with one or more tokens to produce a tokenized database response. The systems and methods provide the tokenized database response to a large language model (LLM). The systems and methods receive a tokenized LLM output that includes at least one of the one or more tokens. The systems and methods produce a detokenized LLM output by replacing the one or more tokens in the tokenized LLM output with the one or more data elements.

Computer-implemented methods and systems for training an adversarial neural network to simulate a stealthy blind false data injection attack on a cyber physical system are provided. Supervised learning is used to generate an initial attack vector, by an adversarial attack generation model, based on inferred grid topology and historical measurements. A final attack vector is generated, by an adversarial verification model, based on a filtered subset of the initial attack vector utilizing a substitute bad data detection threshold, wherein the final attack vector enables creation of a counter measure.

A method and system for generating a privilege based segmented instruction prompt has been developed. Trusted instructions defining the trusted instructions as having a first privilege level, program instructions as having a second privilege level, and data instructions as having a third privilege level are received. The program instructions to implement tasks associated with the data instructions are received. The data instructions are received. The generated privilege based segmented instruction prompt includes the trusted instructions, the program instructions, and the data instructions. The privilege based segmented instruction prompt enables a generative LLM to determine whether the privilege based segmented instruction prompt is an instruction injection attack based on whether there is a conflict between the trusted instructions, the program instructions, and the data instructions in violation of the first, second, and third privilege levels.

A computing system receives a prompt to be provided as input to a large language model. The computing system generates generating an input string to the large language model by appending a plurality of contexts to the prompt. The plurality of contexts defines rules for the large language model to follow when generating the prompt. The plurality of contexts includes a negative context. Based on the prompt and the plurality of contexts, the computing system generates an attention matrix representing relationships between the prompt and the plurality of contexts. The computing system provides the attention matrix to a trained neural network to determine a likelihood that the prompt is malicious. Responsive to determining that the prompt is likely a malicious prompt, the computing system initiates a remedial action.

The embodiment of the invention discloses a prompt injection defense method and system, electronic equipment and a storage medium, and the method comprises the steps: carrying out the preliminary detection of an input text, and obtaining a preliminary detection result; performing semantic analysis on the input text to generate a semantic understanding analysis report; obtaining current operation parameters of the target large language model; according to the current operation parameters, obtaining a behavior pattern analysis result of the target large language model; obtaining a current defense strategy of the target large language model according to the preliminary detection result, the semantic comprehension analysis report and the behavior pattern analysis result; according to the method, the input text and the running state of the large language model are closely combined, so that the prompt injection attack suffered by the large model can be accurately detected, the large model can be efficiently prevented from being hostile attacked, and the corresponding defense strategy can be formulated in a complex attack scene; more effective defense is realized, and the robustness of the whole defense system is improved.

Systems and methods for detecting and mitigating prompt injection attacks on a generative LLM are disclosed. A deployment scenario is considered, in which the generative LLM supports a task automation function. Prompts are received and interpreted by the generative LLM, and outputs from the generative LLM are used to trigger automation actions. The prompts are constructed based on a combination of user input and external data and are, therefore, vulnerable to prompt injection attacks though manipulation of the external data. To mitigate this risk, a separate discriminative classification, decoupled from the generative LLM, engine is configured to identify malicious prompts, and filter out any malicious prompts before they reach the generative LLM.

The invention discloses a power system false data injection attack detection method and system. Measurement data of a power distribution system are acquired; constructing an attack sequence, and injecting the constructed attack sequence into the obtained measurement data to simulate a false data attack; constructing a false data injection attack detection model based on the expansion causal convolution and the recurrent neural network to analyze the state change of the power distribution system; and judging an attack behavior according to the analyzed state of the power distribution system. According to the method, time sequence data can be processed, the long-term dependency relationship and the complex mode can be captured, and the response capability to the complex attack mode can be improved, so that the false data injection attack can be effectively detected and prevented, and the safety and reliability of the power distribution system can be maintained.

This disclosure describes utilizing an attack defense system to improve the defense robustness of a targeted large generative model (LGM) by generating a set of variant prompt injection attacks that are successful against the targeted LGM, where the set of variants is based on a prompt injection attack (e.g., jailbreak) against the targeted LGM or another LGM. For example, the attack defense system utilizes a two-phase framework to generate variant prompt injection attacks and evaluate their attack effectiveness against a targeted LGM. The attack defense system achieves improved variant prompt injection attacks by repeating the two-phase framework and gaining insights from the effectiveness scores of previously generated variants. In addition to generating enhanced variants, the attack defense system generates diverse variants to safeguard the targeted LGM against a broader range of prompt injection attacks that employ more creative and complex styles.

The invention relates to a method for obtaining a hint injection attack test case for a large language model, which combines a conditional variation auto-encoder cVAE and a Markov chain, and obtains a large language model test case through data generation and context modeling, gradual exposure of malicious instructions and simulation of multiple rounds of dialogue attacks in reality. Realizing multiple rounds of dialogue attacks on the large language model, and challenging the defense capability of the large language model; the design scheme introduces a concealment technology, role play attack, state transition and other technologies, improves the complexity and concealment of attacks, has the core advantages of automation, higher concealment, wide coverage, batch testing and the like, can evaluate the security defense capability of the large language model more truly and comprehensively, and improves the security defense capability of the large language model. The defects of an existing defense mechanism are found, and research on multi-round prompt injection attacks and improvement of a security defense mechanism are promoted.

The invention relates to the field of artificial intelligence and natural language processing, in particular to a method, a model and equipment for identifying a prompt injection attack for a large language model. The method comprises the following steps: acquiring a cue word to be input into an LLM; performing semantic analysis on the cue word to obtain a semantic analysis result; performing prompt injection word detection on the prompt word to obtain a prompt injection word detection result; and according to a semantic analysis result and a prompt injection word detection result, analyzing whether the prompt word relates to a prompt injection attack. The invention provides a fusion recognition mechanism combining semantic analysis and prompt injection word detection, and the fusion recognition mechanism is specially used for recognizing the prompt injection attack aiming at the LLM, so that the defense capability for the LLM prompt injection attack is improved.

Disclosed is a training data generation system for generating training data used to train machine learning models to inspect GenAI traffic to identify security and privacy concerns related to GenAI use. The training data generation system is seeded with initial prompts. The initial prompts include benign prompts, prompt injection attacks, and uploaded files. Each initial prompt is submitted to multiple GenAI applications to obtain responses. The corresponding prompts and responses are stored in a training data repository. Variations of the initial prompts are generated using, for example, one of the GenAI applications. Each variation is submitted to each of the GenAI applications as well, and the corresponding prompts and responses are stored. Another machine learning model, regex patterns, a combination, or the like may be used to label the prompts and responses in the training data repository to generate a large training data set quickly and efficiently.

The embodiment of the invention relates to the field of artificial intelligence, and provides a model context protocol injection attack protection method and device based on dynamic semantic analysis, computer equipment and a storage medium, and the method comprises the steps: obtaining request data corresponding to a call request initiated by a user through a model context protocol, carrying out the preprocessing of the request data, obtaining the preprocessed request data; performing semantic vectorization on request text and context historical information in the request data through a lightweight bidirectional encoder representation model to output an initial risk score; correcting the initial risk score according to context historical information carried in a model context protocol to obtain a corrected final risk score; and performing hierarchical defense decision according to the final risk score, and determining risk grading information corresponding to each piece of request data so as to execute a protection action corresponding to each piece of risk grading information. By adopting the method, the accuracy of identifying the protocol injection attack can be improved.

A system and method for multiple fraud type detection includes anti-injection attack system that has a layered architectural approach that uses a includes combination of different specific models to detect the attacks in combination with image processing techniques, device signals and liveness checks to detect the variety of different types of fraud attacks or repeat fraud attacks. The anti-injection attack system applies the analysis of the tools used to create deepfake, face morph and face swap attacks to define the elements of its layered architecture that can these various types of attacks.

A computer-implemented method is provided that prevents prompt injection attacks against generative models. Input data is received, and a first prompt section is generated from the input data. First, second and third instructions are received, which respectively instruct the generative model to carry out a task based on the first prompt section, inform the generative model of a boundary of the first prompt section, and instruct the generative model to ignore any instructions in present in the first prompt section. A prompt for the generative model is generated from the first prompt section and the first instructions, second instructions and third instructions.

The invention discloses a multi-modal attack identification method fusing BMama and difference to guide trans-attention, which comprises the following steps: simulating a false data injection attack, a denial of service attack, an address resolution protocol spoofing attack and a domain name system spoofing attack, collecting physical layer sensor data and network layer flow data, and preprocessing multi-modal data; bMama is constructed to perform dynamic time modeling on multi-modal data, a graph neural network is combined to adversariate a variational auto-encoder, features of a power grid system topology and a communication topology structure are fused, and robustness of potential representation is enhanced through adversarial training; the method comprises the following steps of: guiding feature complementary fusion by using modal difference through a difference guide iteration cross-attention fusion mechanism, improving the capability of distinguishing complex attacks, finally carrying out attack detection and classification on fused modals, and executing end-to-end optimization according to a weighted combination of loss of each part. The method can effectively detect and classify the multi-modal attack in the smart power grid, and enhances the safety and reliability of a complex system.

The invention discloses a power network attack chain dynamic deduction and intelligent response process method, system and device based on deep reinforcement learning and a medium, and belongs to the technical field of network security and power system protection. Multi-modal data is aligned and normalized, an event view cache is constructed, and the generalization detection capability on process camouflage and memory injection attacks is improved through a federated learning collaborative detection mechanism; based on the event view cache and historical threat intelligence, generating a dynamic attack knowledge graph, constructing a deep reinforcement learning model taking the attack knowledge graph as an environment, calculating an attack influence index by using a Bayesian network, and generating a differentiated security response instruction; and realizing attack path backtracking and attack source positioning based on the attack knowledge graph. According to the method, multi-modal data fusion analysis and strategy adaptive updating are realized, and the attack chain identification accuracy and evidence chain construction integrity are remarkably improved.

The invention discloses an electric power system data driving damping controller under false data injection attack, and relates to the technical field of novel electric power system control and network security, and the electric power system data driving damping controller comprises the following steps: S1, aiming at a wide area measurement signal of a damping controller, determining a condition that an attack signal reaches system stability destroying, and constructing multiple types of false data injection models; s2, designing an attack detection mechanism and an attack compensation module based on a threshold value according to the attacked controller measurement signal; s3, establishing a data driving model corresponding to the controlled system under the action of attack compensation; according to the method, the output signal of the controller is updated and the output of the controlled power system at the next moment is predicted by using the online I / O data of the controlled power system, so that the method can adapt to various operation conditions and is high in self-adaptability; the method has the capability of resisting false data injection attacks, the influence of the communication risk of network attacks on the system stability can be avoided, and the robustness of the controller is improved.

Techniques for liveness detection using a motion, face, and context cues. The techniques can be implemented to prevent against successful presentation attacks, video injection attacks, and deepfake attacks. In some examples, the techniques encompass receiving a set of video frames from a personal computing device. A first liveness determination can be made using a motion-based model based on the received video frames. A second liveness determination can be made using a face-based model based on the received video frames. A third liveness determination can be made using a context-based model based on the received video frames. A final liveness determination can be made based on the first, second and third liveness determinations.

The invention provides a direct current micro-grid false data injection attack detection method based on deep learning, and aims to improve the safety and stability of a direct current micro-grid when the direct current micro-grid encounters hostile attacks. According to the method, a parallel double-branch model in which Transform and GRU are combined is adopted, the advantages of the Transform and the GRU are brought into full play, and efficient identification of the false data injection attack (FDIA) is achieved. Through multi-level feature fusion and a decision-making mechanism, the model can accurately identify various false data injection attacks in a complex attack scene, and relatively high detection accuracy is kept. Experimental results show that the method shows excellent performance in a direct current micro-grid environment, has a wide application prospect, and particularly has a remarkable application value in engineering practice.

The invention discloses a false data injection attack detection method and system based on elastic energy prediction. The method comprises the steps that a distributed node network is constructed, nodes are deployed to collect real-time energy output data, each node generates a state estimation value according to a Kalman consensus filtering model, an innovation vector is calculated, and a monitoring threshold is adjusted by combining an association weight of a network topology; inputting a state estimation value, an innovation vector and real-time energy output data, calculating an accumulated deviation trend by using a time sequence analysis method, identifying a slow penetration attack in combination with a sliding window, and judging whether a false data injection attack exists or not; when it is judged that the false data injection attack exists, operation parameters of the energy equipment are adjusted, attacked nodes are isolated, non-attacked nodes are redistributed, and state change information is recorded; and according to the attack frequency, the node association weight and the operation cost, the detection parameters are adjusted in the dual-time scale, the resource allocation weight is optimized based on the feedback data, and the detection probability, the response time and the resource allocation strategy are dynamically updated.

In an embodiment, a method includes receiving a prompt provided to a large language model (LLM), generating a semantic fingerprint for the prompt based on semantic and syntactic features associated with the prompt, generating a vector representation incorporating the semantic fingerprint for the prompt, calculating semantic distances between the vector representation and multiple vector representations associated with multiple malicious intents, determining an intent associated with the prompt based on the semantic distances, and determining a defense action on the prompt based on the intent and multiple policies to mitigate a security risk associated with the prompt.

The invention discloses a hidden false data injection attack detection and security control method for a quad-rotor unmanned aerial vehicle, and belongs to the technical field of information security. The problem that the hidden false data injection attack in the quad-rotor unmanned aerial vehicle system cannot be detected by the existing attack detection method is solved. According to the method, a watermark signal sequence and an auxiliary function are introduced to enable a residual signal to be sensitive to attacks, a residual signal calculation method based on linearization model observability and a parallel Kalman filter is provided, and hidden false data injection attack detection is further carried out on the residual signal. And control input is calculated based on a state estimation result of the parallel Kalman filter, so that safety control of the quad-rotor unmanned aerial vehicle is realized. The method can be applied to hidden false data injection attack detection and security control.

The embodiment of the invention provides a coupling network asynchronous dynamic intermittent safety control method and system, and the method comprises the steps: obtaining multiple attack signals comprising an injection attack signal and a replacement attack signal after building a dynamic model of a following network, a target network and a state error variable; and setting an intermittent controller according to the dynamic network model and the multiple attack signals, and setting a dynamic mechanism based on an exponential function to convert a state error variable into an interval control function. Wherein the interval control function is used for realizing security mean square synchronization control between the following network model and the target network model. According to the method, the asynchronous anti-attack intermittent dynamic cooperative control strategy oriented to the complex coupled network can be constructed based on an average intermittent control rate and dynamic event trigger threshold coupling method by utilizing a node self-adaptive dual spoofing attack probability model, and the problems of difficulty in multi-attack modeling and insufficient control strategy collaboration can be relieved.

The invention discloses a false data injection attack detection method based on measurement physical recovery, and relates to the technical field of network security, and the method comprises the steps: obtaining a current abnormal data set, different from a normal data set, of a detection object; obtaining a historical abnormal data set of the detection object, and determining a threshold range used for detecting the current abnormal data set through a difference value between the historical abnormal data set and the normal data set; and executing a first repair action on the current abnormal data set according to the threshold range, so that the current abnormal data set falls into the threshold range. The method has the advantages that physical constraint is introduced, physical restoration action is executed on the current abnormal data set, enough accuracy and reliability of current abnormal data set restoration are improved, the current abnormal data set and hidden problems brought by the current abnormal data set to a power system are solved, and the power system reliability is improved. And the accurate capture and timely repair of later abnormal data by the power system can be improved.

Various embodiments provide methods, systems, computer program products, apparatuses, and / or the like for assessing vulnerability of an IC design to fault injection attacks, such as through a security property-driven vulnerability assessment framework for efficiently evaluating faults with respect to certain security properties associated with the IC design. In one embodiment, a method is provided. The method includes generating, using a fault-injection technique specification, one or more fault models describing attributes of one or more faults. The method further includes selecting, using the fault models and executable security properties associated with a design file of an IC design, a fault list identifying a plurality of possible faults for the IC design. The method further includes identifying, based at least in part on performing a fault simulation on the design file with the fault list, critical locations of the IC design. The method further includes implementing protections at the critical locations.

The invention discloses a false data injection attack detection method and device. The method comprises the steps that power grid measurement data are acquired, and the power grid measurement data comprise multiple types of power data; power grid measurement data are analyzed through an ensemble classification detection model, an output vector value is obtained, and the ensemble classification detection model is obtained through machine learning training of multiple sets of data; each group of data in the multiple groups of data comprises historical power grid measurement data and a label for indicating whether the historical power grid measurement data is false data injection attack data; and determining whether the power grid measurement data is attacked or not according to the size of the output vector value, and processing the attacked data. The technical problems that in the prior art, a false data injection attack detection method for power system state estimation is insufficient in fitting capacity, and the polluted power grid state quantity cannot be accurately recognized are solved.

A malicious instructions detection model (“detector”) intercepts augmented prompts destined for a large language model (“LLM”). Each augmented prompt was augmented with data from potentially compromised data sources susceptible to indirect prompt injection attacks. The detector tokenizes / preprocesses sentences in the augmented prompts and is invoked on the tokenized / preprocessed sentences to obtain confidence scores that each sentence comprises malicious instructions. If one or more of the confidence scores is above a threshold, the detector blocks the augmented prompt and generates an alert indicating the blocking and the malicious instructions. Otherwise, the detector communicates the augmented prompt to its intended LLM.

Prompt injection attack detection in responses from large language models includes receiving, at a server from a user device, a user prompt segment to an LLM, generating a LLM prompt from the user prompt segment, sending the LLM prompt to the LLM, and receiving a response from the LLM. Prompt injection attack detection further includes comparing the response to a structured data schema for the response to validate the response, and sending, responsive to validating the response, the response to the user device.

One commonality among most vehicular security attacks reported to date is that they ultimately require write access to the CAN bus. In order to cause targeted and intentional changes in the vehicle behavior, malicious CAN injection attacks require knowledge of the CAN message format. However, since this format is proprietary to OEMs and can differ even among different models of a single make of vehicle, one must manually reverse-engineer the CAN message format of each vehicle they target. To mitigate this difficulty, an automated CAN message translator is presented.

The invention provides a cue word injection protection method and device, equipment and a storage medium, and the method comprises the steps: obtaining input information of a target AI application, detecting the input information, determining whether the input information accords with an input protection strategy of the target AI application or not, and obtaining the input protection strategy based on the configuration information of the target AI application through learning, and determining whether the input information has the risk of cue word injection, and if the input information of the target AI application conforms to the input protection strategy, processing the input information by using the large model of the target AI application to obtain an output result. By the adoption of the technical scheme, after the input information of the target AI application is obtained, the input information is detected through the input protection strategy obtained through learning based on the configuration information of the target AI application so as to determine whether the input information has the risk of cue word injection or not, and therefore cue word injection attacks of the target AI application are effectively prevented.