A Greedy Coordinate Gradient Jailbreak Attack Method Based on Spatial Momentum for Large Language Model Optimization
By establishing multiple gradient sampling spaces in a large language model, using data augmentation algorithms to generate semantically equivalent or neighboring variants, performing multi-space gradient sampling and weighted fusion, the problem of inaccurate gradient estimation in white-box attacks by the greedy coordinate gradient method is solved, the attack success rate and the transferability of adversarial examples are improved, and the security of the model is enhanced.
Patent Information
- Application Number
- CN202611167672.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-03
- Publication Date
- 2026-11-17
AI Technical Summary
Existing greedy coordinate gradient methods suffer from inaccurate gradient estimation in white-box attacks on large language models, are prone to getting trapped in local optima during the optimization process, have low attack success rates, and generate adversarial suffixes with poor transferability to black-box models.
A greedy coordinate gradient jailbreak attack method based on spatial momentum is adopted. By establishing multiple gradient sampling spaces, using data augmentation algorithms to generate semantically equivalent or neighboring variants, performing multi-space gradient sampling and weighted fusion, calculating the comprehensive gradient, optimizing the adversarial suffix sequence to avoid local optima, and generating more general adversarial examples.
It improves the performance and robustness of jailbreak attacks in white-box scenarios, and the generated adversarial examples have higher transferability on black-box models, enabling more thorough red team testing and strengthening the security defenses of large models.
Smart Images

Figure CN123116263A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer security technology, and in particular to a greedy coordinate gradient jailbreak attack method based on spatial momentum for large language model optimization. Background Technology
[0002] With the widespread application of large-scale models in natural language processing, machine translation, code generation, and other fields, their security and alignment issues have become increasingly prominent. To ensure that the content generated by large-scale models complies with human ethics and safety guidelines, the industry employs alignment techniques to fine-tune them, enabling them to identify and reject malicious or harmful user queries, such as those involving the production of dangerous items or the dissemination of false information. However, even with these measures, "jailbreak attacks" against aligned large-scale models still pose a security threat. Jailbreak attacks, through carefully crafted adversarial prompts, induce the model to bypass its security mechanisms and output harmful content that it should otherwise reject. In related technologies, defenders typically use jailbreak attack methods for red-team testing to proactively discover and patch security vulnerabilities to improve model security.
[0003] Among numerous automated jailbreak attack methods, attacks based on white-box models, especially the greedy coordinate gradient method, have demonstrated strong attack potential. The greedy coordinate gradient method effectively misleads the model by iteratively optimizing an adversarial suffix appended to the original malicious query using the model's internal gradient information.
[0004] However, existing greedy coordinate gradient methods and their improvements (such as the greedy coordinate gradient method incorporating time momentum) have significant technical limitations. The fundamental problem lies in the fact that the optimization process of discrete terms occurs in a highly non-smooth optimization environment, leading to inaccurate gradient estimation based on single-point computation. This makes the optimization process prone to getting trapped in local optima, thus limiting the success rate of attacks. Experiments show that the greedy coordinate gradient method achieves only a 40%-50% success rate when attacking robust models such as Llama2-7B-Chat, far below the requirements for practical red team testing attacks. Summary of the Invention
[0005] To address the shortcomings of the existing technologies, this invention provides a greedy coordinate gradient jailbreak attack method, apparatus, device, and medium based on spatial momentum for large language model optimization, employing the following technical solution: Firstly, a greedy coordinate gradient jailbreak attack method based on spatial momentum for large language model optimization is provided, including: S1: Establish multiple gradient sampling spaces to generate different semantically equivalent or neighboring variants of the original input; each gradient sampling space is configured with at least one transformer for performing data augmentation algorithms; S2: Obtain the adversarial suffix of the current iteration in the malicious query, perform multi-space gradient sampling on it in parallel based on multiple gradient sampling spaces, obtain the sampled samples and calculate the sampling gradient of each space, and perform weighted fusion of the sampling gradients of all spaces with the gradient of the original input to obtain the comprehensive gradient. S3: Based on the comprehensive gradient, calculate the candidate word sequence set for each word position in the adversarial suffix sequence, and select the candidate adversarial suffix that minimizes the loss function through batch sampling and evaluation, as the update result of this iteration; S4: Repeat S1 to S3 until the preset termination condition is met.
[0006] Secondly, a greedy coordinate gradient jailbreak attack device based on spatial momentum for large language model optimization is provided, comprising: A multi-gradient sampling space construction module is used to establish multiple gradient sampling spaces for generating different semantically equivalent or neighboring variants of the original input; each gradient sampling space is configured with at least one transformer for performing data augmentation algorithms; The gradient sampling and fusion module is used to obtain the adversarial suffix of the current iteration in the malicious query, perform multi-space gradient sampling on it in parallel based on multiple gradient sampling spaces, obtain sampled samples and calculate the sampling gradient of each space, and perform weighted fusion of the sampling gradients of all spaces with the gradient of the original input to obtain the comprehensive gradient. The coordinate update module is used to calculate the position of each word in the adversarial suffix sequence based on the comprehensive gradient, select a set of candidate word sequences, and select the candidate adversarial suffix that minimizes the loss function through batch sampling and evaluation, which is used as the update result of this iteration. The iteration module is used to repeatedly iterate until a preset termination condition is met.
[0007] Thirdly, an electronic device is provided, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, performs the steps of the method as described in the first aspect.
[0008] Fourthly, a readable storage medium is provided, on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method as described in the first aspect.
[0009] The spatial momentum-based greedy coordinate gradient jailbreak attack method for large language model optimization proposed in this application has the following beneficial effects: This application improves the performance and robustness of jailbreak attacks in white-box scenarios, addressing the problem that existing white-box attack methods such as the greedy coordinate gradient method, while accurate gradient estimation in the non-smooth optimization space of discrete words, easily leads to getting trapped in local minima and has a limited success rate. This application overcomes the non-smoothness of discrete optimization, avoids getting trapped in local optima, and can generate a novel jailbreak attack method with more general adversarial examples, thereby enabling more thorough red team testing, evaluation, and ultimately strengthening of the security defenses of large models.
[0010] The apparatus, electronic device, and readable storage medium corresponding to the greedy coordinate gradient jailbreak attack method based on spatial momentum for large language model optimization can achieve the same technical effect, and will not be described in detail here to avoid duplication. Attached Figure Description
[0011] Figure 1 A schematic flowchart illustrating a greedy coordinate gradient jailbreak attack method based on spatial momentum for large language model optimization provided in this application embodiment; Figure 2 A schematic flowchart illustrating another greedy coordinate gradient jailbreak attack method based on spatial momentum for large language model optimization provided in this application embodiment; Figure 3 This is a schematic flowchart illustrating a greedy coordinate update process based on a comprehensive gradient, provided for an embodiment of this application. Detailed Implementation
[0012] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the technical solutions in the embodiments of this application are clearly described. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art are within the scope of protection of this application.
[0013] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.
[0014] The steps described in this application and the flowcharts in the accompanying drawings are not necessarily strictly executed according to the step numbers; the execution order of the steps can be changed. Furthermore, certain steps can be omitted, multiple steps can be combined into one step, and / or one step can be broken down into multiple steps.
[0015] This specification provides a greedy coordinate gradient jailbreak attack method based on spatial momentum for large language model optimization, and also relates to a greedy coordinate gradient jailbreak attack device based on spatial momentum for large language model optimization, a computer device, and a computer-readable storage medium. The following describes each item in detail with reference to the accompanying drawings and preferred embodiments.
[0016] Please see Figure 1-3 This application provides a greedy coordinate gradient jailbreak attack method based on spatial momentum for large language model optimization, including: Step S1: Establish multiple gradient sampling spaces to generate different semantically equivalent or neighboring variants of the original input; each gradient sampling space is configured with at least one transformer for performing data augmentation algorithms.
[0017] This step constructs a multi-space gradient sampling framework, aiming to establish a multi-dimensional spatial system for gradient sampling and provide a source for spatial momentum. Specifically, five core gradient sampling spaces are defined, and these multiple gradient sampling spaces include at least: candidate space, text space, lexical space, one-hot encoding space, and embedding space. Further: The candidate space stores the set of alternative adversarial suffixes generated in the current and historical iterations that differ from the optimal candidate by a few individual lexical terms.
[0018] The candidate space is a sampling space that stores and manages alternative adversarial suffixes generated in the current and historical iterations that differ from the optimal candidate by a few lexical units; specifically, the following calculations are performed in the candidate space: In the In the next iteration: in, For the neighborhood samples obtained by sampling, For the malicious part of the original sample, This is the adversarial suffix part of the original sample. This refers to the adversarial suffix portion of the neighborhood samples obtained by the sampling algorithm. You can use a random sampling function or a loss-first sampling function. This represents the candidate pool that stores data from the current and previous iterations. It is the first In the next iteration, adversarial suffixes are used with a tokenizer. The sampled malicious queries are encoded to obtain a word sequence. The final candidate space gradient formula is as follows: , in, Given the word sequence of the input sample, For the transformed lexical sequence, the first... Each word element, This represents the one-hot encoding form corresponding to a word element, where This is the cross-entropy function between the model response and the expected guiding prefix.
[0019] The text space is stored at the string level as a set of semantically invariant but word-changing variants generated from the original malicious query and adversarial suffixes.
[0020] The text space refers to the sampling space formed by manipulating the original malicious query and adversarial suffixes at the string level to generate semantically invariant but word-changing variants; specifically, the following computations are performed in the text space: set up This is the original malicious query, in which This indicates the malicious part of the original malicious query. This indicates resistance to the suffix. This represents a string concatenation operation. After applying the transform, we get: in, This represents an unfiltered text transformer. Used to calculate sequence length. Used to encode strings into a sequence of tokens; Use a word segmenter The sampled malicious queries are encoded to obtain a word sequence. Finally, the formula for the text space gradient is obtained as follows: , in This is the cross-entropy function between the model response and the expected guiding prefix.
[0021] The lexical space is stored at the model lexical level as a set of variants generated by changing the order or replacing lexical sequences based on the encoded lexical sequences.
[0022] The lexical space refers to the sampling space formed by directly manipulating the encoded lexical sequence at the model lexical level, generating variants by changing the lexical order or substitution; specifically, the following calculations are performed in the lexical space: in, This is the word sequence of the malicious part of the original sample. The word sequence of the adversarial suffix part of the original sample. This represents an unfiltered lexical transformer. This represents a word sequence concatenation operation. Used to decode a sequence of words into a string; The gradient formula for the final word space is: .
[0023] The one-hot encoding space is stored at the level of the one-hot encoding vector of the word, and a set of neighboring regions is generated by injecting small perturbations.
[0024] One-hot coding space refers to the sampling space formed by operations performed at the level of one-hot coding vectors of words, which explores neighboring regions by injecting small perturbations; specifically, the following calculations are performed in the one-hot coding space: First, the word sequence is converted into a one-hot encoding form. , The embedding process is pre-defined, and a loss function is redefined that takes the embedding vector as input. ,in It is an embedding vector; The malicious question and adversarial suffix are transformed separately, and then multiplied by the embedding weight matrix to obtain the modified embedding vector: The gradient formula for the final one-hot encoding space is: in, It is the embedding weight matrix.
[0025] The embedding space is stored at the word embedding vector level of the model, and a set of continuous semantic neighborhood regions is generated by adding noise to the embedding vectors.
[0026] The embedding space refers to the sampling space formed by operations performed at the word embedding vector level of the model. This involves adding noise to the embedding vectors to explore continuous semantic neighborhood regions. Specifically, the following computations are performed in the embedding space: The gradient formula for the embedding space is: .
[0027] After defining the core gradient sampling space, one or more specific transformers need to be instantiated for each of the above spaces to generate different semantically equivalent or neighboring variants of the original input. These transformers constitute the data augmentation method described in this application. The transformers include at least one of the following: A candidate space transformer is used to perform random sampling or loss-based priority sampling. Random sampling specifically refers to storing and managing an adversarial suffix candidate pool generated from the current and historical iterations. A number of candidate objects are uniformly selected from the pool, regardless of their loss values, to facilitate exploration of different gradient directions. Loss-based priority sampling refers to selecting from the candidate pool... Select the one with the lowest loss value The strategy focuses gradient computation on the most promising (lowest loss) direction in the current candidate set, strengthening and optimizing the most effective perturbation.
[0028] A text space transformer is used to perform character-level, word-level, sentence-level, and / or message-level transformations. Character-level transformations include random character replacement (randomly replacing 1-2 characters with other characters); OCR-based replacement (simulating OCR errors by replacing existing characters with visually similar characters, e.g., 'o' vs. '0', 'I' vs. '1'); and keyboard error replacement (replacing characters with characters from adjacent keyboard keys to simulate typing errors, e.g., 'a' vs. 's', 'k' vs. 'I'). Word-level transformations include synonym replacement (replacing a word with its semantically equivalent word using an embedding vector model); random swapping (randomly swapping the positions of two adjacent words); random deletion (randomly deleting a word from a malicious query); and spelling error replacement (using common spelling errors, e.g., 'receive' vs. 'recieve'). Sentence-level transformations include shuffling sentence order (randomly swapping the positions of two sentences) and sentence paraphrasing (replacing sentences with sentences of the same meaning). Message-level transformations include shuffling message order (randomly swapping the positions of two messages throughout the history).
[0029] A lexical space transformer is used to perform random lexical substitution or cyclic shifting. Random lexical substitution specifically refers to randomly replacing a subset of lexical units in a lexical sequence with other valid lexical units from the vocabulary. Cyclic shifting specifically refers to randomly shifting a lexical sequence. The cyclic displacement is calculated as follows: in, The word sequence of the adversarial suffix part of the original sample. For the last word element, Let be the word at the k-th position.
[0030] A one-hot encoded space transformer is used to perform Gaussian noise injection. It adds small random noise to the one-hot encoded vector.
[0031] An embedding space transformer is used to perform Gaussian noise injection. This adds small amounts of random noise to the word embedding vectors.
[0032] Step S2: Obtain the adversarial suffix of the current iteration in the malicious query, perform multi-space gradient sampling on it in parallel based on multiple gradient sampling spaces, obtain the sampled samples and calculate the sampling gradient of each space, and perform weighted fusion of the sampling gradients of all spaces with the gradient of the original input to obtain the comprehensive gradient.
[0033] This step performs multi-spatial gradient sampling and fusion, using the multi-spatial gradient sampling framework constructed in step S1 to collect and fuse gradients.
[0034] For the adversarial suffix of the current iteration, each transformer defined in step S1 is called in parallel to obtain sampled samples. Each transformer generates one or more variants of the adversarial suffix of the transformed malicious query.
[0035] Further, the sampling gradient of each space is calculated, that is, the gradient of each sample relative to the predefined target response is calculated; specifically including: For each sampled sample, perform forward propagation of the large model to calculate its loss relative to the predefined target response; calculate the gradient of the sampled sample with respect to the embedding vector of the current adversarial suffix lexical through backpropagation; The loss formula for the predefined target response is: in Indicating a malicious question, Indicates opposition to suffixes, For a predefined target response, typically a positive response (e.g., "Of course, this is how to make a bomb"), It is the probability distribution of the output of the large model.
[0036] In some possible implementations, the generated sampled samples are also decoded and re-encoded for verification (mainly for the transformation of the text space and the word space) to ensure that the length of the transformed word sequence remains unchanged and is valid, and to filter out invalid samples and their gradients to prevent the optimization from crashing due to the transformation of the word sequence length.
[0037] Furthermore, the weighted summation method includes the sovereign weight corresponding to the original malicious query loss gradient and the weight corresponding to each spatial gradient. Specifically, the formula for calculating the comprehensive gradient is as follows: in The gradient used to calculate the loss of the target response. Indicates the number of transformations required. Indicates the index of the adversarial suffix sequence. It is the length of the encoded adversarial suffix lexical sequence. This indicates that it corresponds to the index. The one-hot encoded vector of the word sequence, with coefficients and It is a weighting factor that can be preset and adjusted based on the importance of the space or experience.
[0038] Step S3: Based on the comprehensive gradient, calculate the candidate word sequence set for each word position in the adversarial suffix sequence, and select the candidate adversarial suffix that minimizes the loss function through batch sampling and evaluation, as the update result of this iteration.
[0039] Further, in step S3, based on the comprehensive gradient, a candidate lemma sequence set is selected for each lemma position in the adversarial suffix sequence. Through batch sampling and evaluation, the candidate adversarial suffix that minimizes the loss function is selected as the update result of this iteration. Specifically, this includes: Step S301: Using the comprehensive gradient, for each word position in the adversarial suffix sequence, calculate the gradient descent contribution (i.e., negative gradient) of all possible words at that position, and select the K words with the largest contribution for each position to form a candidate word sequence set. Step S302: From the candidate lexical sequence set at all positions, generate B candidate adversarial suffixes by random sampling, combine them with the original malicious query and input them into the target large model to calculate their loss value; in the generated candidate adversarial suffixes, each new suffix is different from the current suffix only at one random position; Step S303: From the B candidate adversarial suffixes, select the one that minimizes the loss function as the adversarial suffix for this iteration update.
[0040] Step S4: Repeat S1 to S3 until a preset termination condition is met. For example, until a preset number of iterations is reached or the attack is successful.
[0041] Based on the above scheme, this application can effectively find better adversarial suffixes in non-smooth discrete optimization spaces, significantly improving the success rate and robustness of attacks in white-box scenarios. It addresses the problem that existing white-box attack methods, such as the greedy coordinate gradient method, suffer from limited success rates in non-smooth optimization spaces of discrete words due to accurate gradient estimation and the resulting susceptibility to local minima. Furthermore, existing methods tend to overfit specific white-box models, resulting in poor transferability to black-box models. This application overcomes the non-smoothness of discrete optimization, avoids getting trapped in local optima, and generates novel jailbreak attack methods with more general adversarial examples, improving transferability to black-box models. This allows for more thorough red-team testing, evaluation, and ultimately, strengthening the security defenses of large models.
[0042] Corresponding to the above-described embodiment of the greedy coordinate gradient jailbreak attack method based on spatial momentum for large language model optimization, this application embodiment provides a greedy coordinate gradient jailbreak attack device based on spatial momentum for large language model optimization, comprising: A multi-gradient sampling space construction module is used to establish multiple gradient sampling spaces for generating different semantically equivalent or neighboring variants of the original input; each gradient sampling space is configured with at least one transformer for performing data augmentation algorithms; The gradient sampling and fusion module is used to obtain the adversarial suffix of the current iteration in the malicious query, perform multi-space gradient sampling on it in parallel based on multiple gradient sampling spaces, obtain sampled samples and calculate the sampling gradient of each space, and perform weighted fusion of the sampling gradients of all spaces with the gradient of the original input to obtain the comprehensive gradient. The coordinate update module is used to calculate the position of each word in the adversarial suffix sequence based on the comprehensive gradient, select a set of candidate word sequences, and select the candidate adversarial suffix that minimizes the loss function through batch sampling and evaluation, which is used as the update result of this iteration. The iteration module is used to repeatedly iterate until a preset termination condition is met.
[0043] Furthermore, the plurality of gradient sampling spaces include at least: a candidate space, a text space, a lexical space, a one-hot encoding space, and an embedding space; The candidate space stores the set of alternative adversarial suffixes generated in the current and historical iterations that differ from the optimal candidate by a few lexical units; The text space is stored at the string level as a set of semantically invariant but word-changing variants generated based on the original malicious query and adversarial suffixes; The lexical space is stored at the model lexical level as a set of variants generated by changing the lexical order or replacing the encoded lexical sequence. The one-hot coding space is stored at the level of the one-hot coding vector of the word, and a set of neighboring regions is generated by injecting small perturbations. The embedding space is stored at the word embedding vector level of the model, and a set of continuous semantic neighborhood regions is generated by adding noise to the embedding vectors.
[0044] Furthermore, the converter includes: Candidate space transformers are used to perform random sampling or loss-based priority sampling. A text space transformer for performing character-level transformations, word-level transformations, sentence-level transformations, and / or message-level transformations; A lexical space transformer is used to perform random lexical substitutions or cyclic shifts. One-hot encoded space transform for performing Gaussian noise injection; An embedded space transformer is used to perform Gaussian noise injection.
[0045] Further, the sampling gradient of each space is calculated, that is, the gradient of each sample relative to the predefined target response is calculated, including: For each sampled sample, perform forward propagation of the large model to calculate its loss relative to the predefined target response; then calculate the gradient of the sampled sample with respect to the embedding vector of the current adversarial suffix lexical through backpropagation.
[0046] Furthermore, the coordinate update module is specifically used for: Using the comprehensive gradient, for each lexical position in the adversarial suffix sequence, calculate the gradient descent contribution of all possible lexical positions at that position, and select the K lexical positions with the largest contributions for each position to form a candidate lexical sequence set. From the set of candidate lexical sequences at all positions, B candidate adversarial suffixes are generated by random sampling. These suffixes are then combined with the original malicious query and input into the target large model to calculate their loss values. In the generated candidate adversarial suffixes, each new suffix differs from the current suffix only at one random position. From the B candidate adversarial suffixes, select the one that minimizes the loss function as the adversarial suffix for this iteration update.
[0047] The above-described greedy coordinate gradient jailbreak attack device based on spatial momentum for large language model optimization implements the steps and processes of the above-described greedy coordinate gradient jailbreak attack method based on spatial momentum for large language model optimization, and achieves the same technical effect. To avoid repetition, it will not be described again here.
[0048] Corresponding to the above-described embodiment of the greedy coordinate gradient jailbreak attack method based on spatial momentum for large language model optimization, this application provides an electronic device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the steps and processes of the above-described embodiment of the greedy coordinate gradient jailbreak attack method based on spatial momentum for large language model optimization, and achieves the same technical effect. To avoid repetition, it will not be described again here.
[0049] Memory can be used to store software programs and various data. Memory can primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area can store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, memory can include volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (Synchlink DRAM, SLDRAM), and direct memory bus RAM (DRRAM). The memory in the embodiments of this application includes, but is not limited to, these and any other suitable types of memory.
[0050] The processor may include one or more processing units; optionally, the processor integrates an application processor and a modem processor, wherein the application processor mainly handles operations related to the operating system, user interface, and applications, while the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into the processor.
[0051] Corresponding to the above-described embodiment of the greedy coordinate gradient jailbreak attack method based on spatial momentum for large language model optimization, this application embodiment also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the steps and processes of the above-described embodiment of the greedy coordinate gradient jailbreak attack method based on spatial momentum for large language model optimization, and achieve the same technical effect. To avoid repetition, they will not be described again here.
[0052] The processor is the processor in the electronic device described in the above embodiments of this application. The readable storage medium includes a computer-readable storage medium, such as a computer read-only memory (ROM), random access memory (RAM), a magnetic disk, or an optical disk.
[0053] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.
[0054] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.
[0055] It is understood that the embodiments of this application have been described above in conjunction with the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. As those skilled in the art will know, various changes or equivalent substitutions can be made to these features and embodiments without departing from the spirit and scope of the invention. Furthermore, those skilled in the art, under the guidance or instruction of this application, can modify these features and embodiments to adapt to specific situations and materials without departing from the spirit and scope of the invention. Therefore, this invention is not limited to the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the protection scope of this invention.
Claims
1. A greedy coordinate gradient jailbreak attack method based on spatial momentum for large language model optimization, characterized in that, include: S1: Establish multiple gradient sampling spaces to generate different semantically equivalent or neighboring variants of the original input; each gradient sampling space is configured with at least one transformer for performing data augmentation algorithms; S2: Obtain the adversarial suffix of the current iteration in the malicious query, perform multi-space gradient sampling on it in parallel based on multiple gradient sampling spaces, obtain the sampled samples and calculate the sampling gradient of each space, and perform weighted fusion of the sampling gradients of all spaces with the gradient of the original input to obtain the comprehensive gradient. S3: Based on the comprehensive gradient, calculate the candidate word sequence set for each word position in the adversarial suffix sequence, and select the candidate adversarial suffix that minimizes the loss function through batch sampling and evaluation, as the update result of this iteration; S4: Repeat S1 to S3 until the preset termination condition is met.
2. The greedy coordinate gradient jailbreak attack method based on spatial momentum for large language model optimization according to claim 1, characterized in that, The plurality of gradient sampling spaces include at least: candidate space, text space, lexical space, one-hot encoding space, and embedding space; The candidate space stores the set of alternative adversarial suffixes generated in the current and historical iterations that differ from the optimal candidate by a few lexical units; The text space is stored at the string level as a set of semantically invariant but word-changing variants generated based on the original malicious query and adversarial suffixes; The lexical space is stored at the model lexical level as a set of variants generated by changing the lexical order or replacing the encoded lexical sequence. The one-hot coding space is stored at the level of the one-hot coding vector of the word, and a set of neighboring regions is generated by injecting small perturbations. The embedding space is stored at the word embedding vector level of the model, and a set of continuous semantic neighborhood regions is generated by adding noise to the embedding vectors.
3. The greedy coordinate gradient jailbreak attack method based on spatial momentum for large language model optimization according to claim 2, characterized in that, The converter includes: Candidate space transformers are used to perform random sampling or loss-based priority sampling. A text space transformer for performing character-level transformations, word-level transformations, sentence-level transformations, and / or message-level transformations; A lexical space transformer is used to perform random lexical substitutions or cyclic shifts. One-hot encoded space transform for performing Gaussian noise injection; An embedded space transformer is used to perform Gaussian noise injection.
4. The greedy coordinate gradient jailbreak attack method based on spatial momentum for large language model optimization according to claim 1, characterized in that, Calculating the sampling gradient for each space, i.e., calculating the gradient of each sample relative to the predefined target response, includes: For each sampled sample, perform forward propagation of the large model to calculate its loss relative to the predefined target response; then calculate the gradient of the sampled sample with respect to the embedding vector of the current adversarial suffix lexical through backpropagation.
5. The greedy coordinate gradient jailbreak attack method based on spatial momentum for large language model optimization according to claim 1, characterized in that, Step S3 specifically includes: Using the comprehensive gradient, for each lexical position in the adversarial suffix sequence, calculate the gradient descent contribution of all possible lexical positions at that position, and select the K lexical positions with the largest contributions for each position to form a candidate lexical sequence set. From the set of candidate lexical sequences at all positions, B candidate adversarial suffixes are generated by random sampling. These suffixes are then combined with the original malicious query and input into the target large model to calculate their loss values. In the generated candidate adversarial suffixes, each new suffix differs from the current suffix only at one random position. From the B candidate adversarial suffixes, select the one that minimizes the loss function as the adversarial suffix for this iteration update.
6. A greedy coordinate gradient jailbreak attack device based on spatial momentum for large language model optimization, characterized in that, include: A multi-gradient sampling space construction module is used to establish multiple gradient sampling spaces for generating different semantically equivalent or neighboring variants of the original input; each gradient sampling space is configured with at least one transformer for performing data augmentation algorithms; The gradient sampling and fusion module is used to obtain the adversarial suffix of the current iteration in the malicious query, perform multi-space gradient sampling on it in parallel based on multiple gradient sampling spaces, obtain sampled samples and calculate the sampling gradient of each space, and perform weighted fusion of the sampling gradients of all spaces with the gradient of the original input to obtain the comprehensive gradient. The coordinate update module is used to calculate the position of each word in the adversarial suffix sequence based on the comprehensive gradient, select a set of candidate word sequences, and select the candidate adversarial suffix that minimizes the loss function through batch sampling and evaluation, which is used as the update result of this iteration. The iteration module is used to repeatedly iterate until a preset termination condition is met.
7. The greedy coordinate gradient jailbreak attack device based on spatial momentum for large language model optimization according to claim 6, characterized in that, The plurality of gradient sampling spaces include at least: candidate space, text space, lexical space, one-hot encoding space, and embedding space; The candidate space stores the set of alternative adversarial suffixes generated in the current and historical iterations that differ from the optimal candidate by a few lexical units; The text space is stored at the string level as a set of semantically invariant but word-changing variants generated based on the original malicious query and adversarial suffixes; The lexical space is stored at the model lexical level as a set of variants generated by changing the lexical order or replacing the encoded lexical sequence. The one-hot coding space is stored at the level of the one-hot coding vector of the word, and a set of neighboring regions is generated by injecting small perturbations. The embedding space is stored at the word embedding vector level of the model, and a set of continuous semantic neighborhood regions is generated by adding noise to the embedding vectors.
8. The greedy coordinate gradient jailbreak attack device based on spatial momentum for large language model optimization according to claim 6, characterized in that, The converter includes: Candidate space transformers are used to perform random sampling or loss-based priority sampling. A text space transformer for performing character-level transformations, word-level transformations, sentence-level transformations, and / or message-level transformations; A lexical space transformer is used to perform random lexical substitutions or cyclic shifts. One-hot encoded space transform for performing Gaussian noise injection; An embedded space transformer is used to perform Gaussian noise injection.
9. An electronic device, characterized in that, The electronic device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, it implements the steps of the spatial momentum-based greedy coordinate gradient jailbreak attack method for large language model optimization as described in any one of claims 1 to 5.
10. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the spatial momentum-based greedy coordinate gradient jailbreak attack method for large language model optimization as described in any one of claims 1 to 5.
