Large model adversarial sample pair generation method and device, electronic equipment and storage medium

Abstract

The embodiment of the application provides a large model adversarial sample pair generation method and device, electronic equipment and storage medium, relating to the technical field of multi-modal large model adversarial samples. The method comprises: inputting an image-text pair into a target large model in a loop to obtain a corresponding hidden state; inputting the hidden state into a probability prediction model to obtain a predicted jailbreak probability; updating the image-text pair based on the predicted jailbreak probability until a first predetermined end condition is reached, ending the update, and obtaining an adversarial sample pair. The embodiment of the application updates the image-text pair through the jailbreak probability of the hidden state, thereby generating an adversarial sample pair, which can effectively avoid direct conflict with large model security alignment and improve the generation efficiency of the adversarial sample pair.

Description

Technical Field

[0001] This application relates to the field of multimodal large-model adversarial sample technology. Specifically, this application relates to a method, apparatus, electronic device, computer-readable storage medium, and computer program product for generating large-model adversarial sample pairs. Background Technology

[0002] With the rapid development of artificial intelligence technology, large multimodal models have demonstrated excellent performance on various multimodal tasks. However, large multimodal models are also susceptible to attacks from adversarial examples. By adding special perturbations to an originally clean image, deep neural networks can make incorrect predictions. These images with added perturbations are called adversarial examples.

[0003] In existing technologies, multimodal jailbreak attacks based on adversarial attacks typically achieve jailbreaks by aligning the model output with harmful text. This approach directly conflicts with secure alignment, has high time costs, and low efficiency in generating adversarial examples. Summary of the Invention

[0004] This application provides a method, apparatus, electronic device, computer-readable storage medium, and computer program product for generating large-scale adversarial sample pairs, aiming to solve the technical problem of low efficiency in existing adversarial sample generation methods.

[0005] Firstly, a method for generating adversarial example pairs for large models is provided, which includes:

[0006] The image and text pairs are repeatedly input into the target large model to obtain the corresponding hidden states;

[0007] Input the hidden state into the probability prediction model to obtain the predicted jailbreak probability;

[0008] Based on the predicted jailbreak probability, the image-text pair is updated until the first predetermined termination condition is met, at which point the update ends and an adversarial sample pair is obtained; the adversarial sample pair is the updated image-text pair.

[0009] Optionally, the target large model includes multiple transformation modules, each of which outputs a hidden state. There are multiple probabilistic prediction models, and each probabilistic prediction model corresponds one-to-one with a transformation module.

[0010] The hidden state is input into the probability prediction model to obtain the predicted jailbreak probability, including:

[0011] Each hidden state is input into the corresponding probability prediction model to obtain the corresponding predicted jailbreak probability.

[0012] Optionally, based on the predicted jailbreak probability, the image-text pair is updated until a first predetermined termination condition is met, at which point the update ends, resulting in adversarial example pairs, including:

[0013] Based on each predicted jailbreak probability, update the image-text pair until the first predetermined termination condition is met, at which point the update ends and an adversarial sample pair is obtained.

[0014] Optionally, based on each predicted jailbreak probability, update the image-text pairs until a first predetermined termination condition is met, at which point the update ends, resulting in adversarial example pairs, including:

[0015] Based on the predicted jailbreak probability and target probability, determine the mean squared error loss function;

[0016] Based on the mean squared error loss function, the parameters of the image-text pair are updated until the first predetermined termination condition is met, at which point the update ends and an adversarial example pair is obtained.

[0017] Optionally, based on the predicted jailbreak probability, the image-text pair is updated until a first predetermined termination condition is met, at which point the update ends, resulting in adversarial example pairs, including:

[0018] Determine the jailbreak loss function based on the predicted jailbreak probability;

[0019] The parameters of the image-text pair are updated based on the jailbreak loss function until the first predetermined termination condition is met, at which point the update ends and an adversarial sample pair is obtained.

[0020] Optionally, the parameters of the image-text pair are updated based on the jailbreak loss function until a first predetermined termination condition is met, at which point the update ends, resulting in adversarial example pairs, including:

[0021] The parameters of the image-text pair are updated based on the jailbreak loss function, and the number of iterations for the image-text pair is determined.

[0022] If the number of iterations reaches the first predetermined termination condition, the update ends, and the adversarial sample pair is obtained.

[0023] Optionally, obtain a training sample set; the training sample set includes multiple training samples.

[0024] The training samples are input one by one into the target large model to obtain multiple corresponding response results;

[0025] Based on multiple responses, determine the corresponding reference jailbreak probability;

[0026] Obtain the hidden state of the target large model for the training samples, input the hidden state into the initial prediction model, and obtain the predicted jailbreak probability.

[0027] Based on the reference jailbreak probability and the predicted jailbreak probability, the parameters of the initial probability model are updated until the second predetermined termination condition is met, at which point the training ends and a well-trained probability prediction model is obtained.

[0028] Optionally, based on multiple response results, a corresponding reference jailbreak probability can be determined, including:

[0029] Based on the preset content evaluation components, determine the content evaluation result of each response;

[0030] Based on the content evaluation results, the corresponding reference jailbreak probability is determined.

[0031] Optional, content assessment results may include harmful content;

[0032] Based on the content evaluation results, the corresponding reference jailbreak probability is determined, including:

[0033] The number of responses deemed harmful by the content assessment will be used as the first number;

[0034] The reference jailbreak probability is determined based on the number of response results and the first number.

[0035] Optionally, the second predetermined termination condition includes the probability difference being less than or equal to a preset difference threshold;

[0036] Based on the reference jailbreak probability and the predicted jailbreak probability, the parameters of the initial probability model are updated until a second predetermined termination condition is met, at which point training ends, resulting in a trained probability prediction model, including:

[0037] The absolute value of the difference between the predicted jailbreak probability and the reference jailbreak probability is used as the probability difference.

[0038] The parameters of the initial probability model are updated based on the probability difference until the probability difference is less than or equal to a preset difference threshold, at which point the training ends and a well-trained probability prediction model is obtained.

[0039] Secondly, a large-model adversarial example pair generation device is provided, the device comprising:

[0040] The state acquisition module is used to repeatedly input image and text pairs into the target large model to obtain the corresponding hidden states;

[0041] The probability prediction module is used to input each hidden state into the probability prediction model to obtain the predicted probability of jailbreak.

[0042] The sample generation module is used to update the image-text pair based on the predicted jailbreak probability until a first predetermined termination condition is met, at which point the update ends and an adversarial sample pair is obtained; the adversarial sample pair is the updated image-text pair.

[0043] Thirdly, an electronic device is provided, the electronic device comprising:

[0044] A memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of any of the methods in the first aspect of this application.

[0045] Fourthly, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the large model adversarial example pair generation method shown in any of the first aspects of this application.

[0046] Fifthly, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the steps of any one of the methods in the first aspect of this application.

[0047] The beneficial effects of the technical solutions provided in this application are:

[0048] The large-model adversarial example pair generation method provided in this application obtains the corresponding hidden states by cyclically inputting image-text pairs into the target large model. The hidden states are then input into a probability prediction model to obtain the predicted jailbreak probability. Based on the predicted jailbreak probability, the image-text pairs are updated until a termination condition is met, resulting in adversarial example pairs. By associating the hidden states with the jailbreak probability and using the predicted jailbreak probability to guide the updating of the image-text pairs, adversarial examples with stronger jailbreak capabilities are obtained. This method does not require offsetting the influence of security alignment, reduces the number of iterations for generating adversarial example pairs, shortens the time, and improves the generation efficiency of adversarial example pairs.

[0049] Furthermore, based on the output of the large model, the reference jailbreak probability of the training samples is determined, the corresponding hidden state is obtained, and the hidden state is input into the initial prediction model to obtain the predicted jailbreak probability. Based on the reference jailbreak probability and the predicted jailbreak probability, the parameters of the initial probability model are updated until the predetermined termination condition is reached, and the trained probability prediction model is obtained. By constructing the relationship between the hidden state and the jailbreak probability through the probability prediction model, the image-text pair can be updated quickly based on the predicted jailbreak probability, which effectively improves the generation efficiency of adversarial example pairs.

[0050] In addition, the target large model can include multiple transformation modules, each of which outputs a hidden state and a probability prediction model. The hidden state is input into the corresponding probability prediction model to obtain the predicted jailbreak probability corresponding to the hidden state. The mean squared error loss function is determined by multiple predicted jailbreak probabilities, thereby updating the image-text pair. The hidden states of each transformation module are fully considered, making the updated adversarial example pair have a stronger jailbreak capability. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below.

[0052] Figure 1 This is a schematic diagram illustrating an application scenario of a large model adversarial example pair generation method provided in an embodiment of this application;

[0053] Figure 2 A flowchart illustrating a method for generating large model adversarial sample pairs provided in an embodiment of this application;

[0054] Figure 3 A schematic diagram of an image in a large model adversarial example pair generation method provided in an embodiment of this application;

[0055] Figure 4 A flowchart illustrating the generation of a probability prediction model in a large model adversarial example pair generation method provided in this application embodiment;

[0056] Figure 5 A flowchart illustrating an example of a large model adversarial example pair generation method provided in this application embodiment;

[0057] Figure 6 This is a schematic diagram of the structure of a large model adversarial sample pair generation device provided in an embodiment of this application;

[0058] Figure 7 This is a schematic diagram of the structure of an electronic device for which a large model adversarial example generation method is applicable, as provided in an embodiment of this application. Detailed Implementation

[0059] The embodiments of this application are described below with reference to the accompanying drawings. It should be understood that the embodiments described below with reference to the accompanying drawings are exemplary descriptions for explaining the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions of the embodiments of this application.

[0060] Those skilled in the art will understand that, unless otherwise stated, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the terms “comprising” and “including” as used in the embodiments of this application mean that the corresponding feature can be implemented as the presented feature, information, data, step, operation, element, and / or component, but do not exclude implementation as other features, information, data, step, operation, element, component, and / or combinations thereof supported by the art. It should be understood that when we say that an element is “connected” or “coupled” to another element, the one element can be directly connected or coupled to the other element, or it can mean that the one element and the other element establish a connection relationship through an intermediate element. Furthermore, “connected” or “coupled” as used herein can include wireless connection or wireless coupling. The terms “or,” “and / or,” “including at least one of the following,” etc., as used in this application, can be interpreted as inclusive, or mean any one or any combination thereof. For example, "including at least one of the following: A, B, C" means "any one of the following: A; B; C; A and B; A and C; B and C; A and B and C", and "A, B or C" or "A, B and / or C" means "any one of the following: A; B; C; A and B; A and C; B and C; A and B and C".

[0061] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0062] In the specific embodiments of this application, any data related to the object, such as data involved in the use of the application, is required. When the embodiments of this application are applied to specific products or technologies, permission or consent from the object is required, and the collection, use, and processing of the relevant data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. In other words, if any of the aforementioned object-related data is involved in the embodiments of this application, this data must be obtained with the object's authorization and consent, and in accordance with the relevant laws, regulations, and standards of the country and region.

[0063] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0064] First, the technical terms used in this application will be introduced and explained:

[0065] The Transformer module is a basic building block in the Transformer architecture, a neural network model based on the self-attention mechanism. It is a deep learning component used to process sequential data. The Transformer module mainly consists of two parts: the self-attention mechanism and the feedforward neural network.

[0066] Hidden state: In large language models, hidden state usually refers to the changes in the internal state of the model when processing input data. These states include, but are not limited to, contextual information or hidden layer outputs. Hidden state is maintained through a self-attention mechanism. Each transformation module processes the input and generates a hidden state, which is then used as the input for the next block.

[0067] Jailbreak attacks are a type of attack targeting machine learning models. Their purpose is to make the model make incorrect predictions that the attacker desires. These attacks typically involve adversarial examples.

[0068] Adversarial examples: In adversarial machine learning, attackers generate adversarial examples. These examples are carefully designed to cause the model to behave incorrectly in the model's input space. If the model makes incorrect predictions about these adversarial examples, it can be considered that the model has been "jailbroken" to some extent.

[0069] In existing technologies, jailbreaking attacks targeting multimodal large models mainly include two implementation methods: visual cue injection and adversarial attacks. Visual cue injection involves embedding jailbreaking commands into the visual modality, such as exploiting the relatively poor security alignment of the visual modality. Adversarial attacks achieve jailbreaking by setting a specific jailbreaking-related loss function and maximizing this loss. Existing adversarial attack-based multimodal jailbreaking attacks typically achieve jailbreaking by aligning the model output with harmful text. However, this approach directly conflicts with secure alignment, as secure alignment aligns harmful inputs with harmless responses, which is exactly the opposite of the goal of the aforementioned jailbreaking attacks. Therefore, the aforementioned jailbreaking attacks require a large number of iterations (usually thousands of iterations) and a large perturbation range to offset the influence of secure alignment. The generation time for adversarial examples is long, resulting in low efficiency.

[0070] The large-model adversarial example generation method, apparatus, electronic device, computer-readable storage medium, and computer program product provided in this application are intended to solve at least one of the above-mentioned technical problems in the prior art.

[0071] To address at least one of the aforementioned technical problems or areas requiring improvement in related technologies, this application proposes a method, apparatus, electronic device, computer-readable storage medium, and computer program product for generating large-scale adversarial sample pairs. The method provides a method for generating large-scale adversarial sample pairs by iteratively inputting image-text pairs into a target large-scale model to obtain corresponding hidden states. These hidden states are then input into a probability prediction model to obtain predicted jailbreak probabilities. Based on these predicted jailbreak probabilities, the image-text pairs are updated until a termination condition is met, resulting in adversarial sample pairs. By associating the hidden states with the jailbreak probabilities and using the predicted jailbreak probabilities to guide the updating of the image-text pairs, adversarial samples with stronger jailbreak capabilities are obtained. This eliminates the need to counteract the effects of security alignment, reduces the number of iterations in adversarial sample generation, shortens the time, and improves the efficiency of adversarial sample pair generation.

[0072] Furthermore, based on the output of the large model, the reference jailbreak probability of the training samples is determined, the corresponding hidden state is obtained, and the hidden state is input into the initial prediction model to obtain the predicted jailbreak probability. Based on the reference jailbreak probability and the predicted jailbreak probability, the parameters of the initial probability model are updated until the predetermined termination condition is reached, and the trained probability prediction model is obtained. By constructing the relationship between the hidden state and the jailbreak probability through the probability prediction model, the image-text pair can be updated quickly based on the predicted jailbreak probability, which effectively improves the generation efficiency of adversarial example pairs.

[0073] In addition, the target large model can include multiple transformation modules, each of which outputs a hidden state and a probability prediction model. The hidden state is input into the corresponding probability prediction model to obtain the predicted jailbreak probability corresponding to the hidden state. The mean squared error loss function is determined by multiple predicted jailbreak probabilities, thereby updating the image-text pair. The hidden states of each transformation module are fully considered, making the updated adversarial example pair have a stronger jailbreak capability.

[0074] The technical solutions of this application and their effects are described below through several exemplary embodiments. It should be noted that the following embodiments can be referenced, borrowed from, or combined with each other. Identical terms, similar features, and similar implementation steps in different embodiments will not be repeated.

[0075] Figure 1 This is a schematic diagram of an application scenario for the large model adversarial sample pair generation method provided in this application embodiment. The application environment may include a terminal 101 or a server, and the terminal 101 is configured with a large model adversarial sample pair generation system.

[0076] Specifically, terminal 101 cyclically inputs the image-text pair into the target large model to obtain the corresponding hidden state, inputs the hidden state into the probability prediction model to obtain the predicted jailbreak probability, updates the image-text pair based on the predicted jailbreak probability, until the first predetermined termination condition is reached, the update ends, and an adversarial sample pair is obtained. The adversarial sample pair is the updated image-text pair.

[0077] The above application scenario is just an example and does not limit the application scenarios of the large model adversarial example generation method in this application.

[0078] Those skilled in the art will understand that the terminal can be a smartphone (such as an Android phone, an iOS phone, etc.), a tablet computer, a laptop computer, a digital broadcast receiver, a MID (Mobile Internet Device), a PDA (Personal Digital Assistant), a desktop computer, a smart home appliance, an in-vehicle terminal (such as an in-vehicle navigation terminal, an in-vehicle computer, etc.), a smart speaker, a smartwatch, etc. The terminal and the server can be connected directly or indirectly through wired or wireless communication, but are not limited to these.

[0079] The server may include servers installed with the ability to handle database operations. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server or server cluster providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. The embodiments of this application can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, smart transportation, and assisted driving. Specific applications can be determined based on actual application needs and are not limited here.

[0080] In some possible implementations, taking a large model adversarial example pair generation system as an example, embodiments of this application provide a method for generating large model adversarial example pairs, such as... Figure 2 As shown, it may include the following steps:

[0081] S210, input the image and text pairs into the target large model in a loop to obtain the corresponding hidden states.

[0082] An image-text pair includes image content and text content. There can be multiple image-text pairs. Multiple image-text pairs may have the same image content but different text content. The image content and text content in an image-text pair may or may not be related.

[0083] Among them, the target large model can be a multimodal large model capable of processing images and text, including language-driven visual analysis models and large-scale visual language models.

[0084] Specifically, image-text pairs are acquired and input into a target large model. This target large model can be a white-box model. The hidden state information generated by the target large model during the processing of the image-text pairs is obtained. The hidden state refers to the changes in the internal state of the model when processing the input data. These states include, but are not limited to, context information or hidden layer outputs.

[0085] In the specific implementation process, based on the preset default images and test text sets, the default images are combined with multiple text data in the test text sets to obtain multiple image-text pairs, thus obtaining the test sample set. The test text set is obtained by dividing the preset adversarial dataset. The adversarial dataset can be a combination of the adversarial sample benchmark set AdvBench, the strong denial attack set StrongREJECT, and the jailbreak benchmark set JailbreakBench. The test text set can also include the mini jailbreak dataset miniJailbreakV 28K, which evaluates the robustness of a multimodal large language model against various jailbreak attacks.

[0086] In the actual implementation process, the image content is pre-set, such as... Figure 3 As shown, the content of the image is not limited to pandas; this is just an example and does not impose any restrictions on the image content or style.

[0087] S220: Input the hidden state into the probability prediction model to obtain the predicted jailbreak probability.

[0088] Specifically, the probabilistic prediction model includes the relationship between the hidden state and the jailbreak probability. The hidden state is input into the probabilistic prediction model to obtain the predicted jailbreak probability output by the model. Based on the predicted jailbreak probability, the parameters of the image-text pair corresponding to the hidden state are updated to obtain adversarial sample pairs.

[0089] S230: Based on the predicted jailbreak probability, update the image-text pair until the first predetermined termination condition is met, then end the update and obtain the adversarial example pair.

[0090] Among them, the adversarial sample pairs are the updated image-text pairs.

[0091] Specifically, the predicted jailbreak probability is compared with 1 (i.e., 100% probability of jailbreak), the difference between the two is determined, and the corresponding image-text pair is updated so that the difference between the predicted jailbreak probability and 1 gradually decreases until the first predetermined termination condition is met, the update ends, and adversarial sample pairs are obtained. There can be multiple image-text pairs. After updating multiple image-text pairs, multiple adversarial sample pairs are obtained, and adversarial samples are generated.

[0092] In the specific implementation process, updating the image-text pair can be done by updating the pixel parameters of the image or updating the text. When updating the image parameters, two constraints are applied. The first is to constrain the pixel value of each pixel to be between 0 and 1 to ensure that the pixel is within the legal range. The second is to constrain the perturbation range, which needs to ensure that the perturbation range is small enough to be imperceptible to the human eye, while also misleading the model, so as to ensure high invisibility against noise.

[0093] In practical implementation, for the features of an image-text pair, such as each pixel of the image in the image-text pair, adjustments are made according to the direction of the gradient. If the gradient is positive, the value of that feature is increased; if the gradient is negative, the value of that feature is decreased. The adjustment magnitude usually needs to be proportional to the magnitude of the gradient and multiplied by a small coefficient, called the learning rate or step size, to ensure that the update perturbation is not too large. Specific image parameter adjustments can include: increasing the pixelation level of the image, changing the color, contrast, brightness of pixels, or adding noise to the image, etc. The methods for adjusting image parameters are not limited to these examples and can be set according to actual needs.

[0094] In the specific implementation process, the first predetermined termination condition may include: accuracy, loss function value reaching a threshold, and iteration count reaching a threshold. Specifically, the first predetermined termination condition may include the accuracy reaching a preset threshold, comparing the predicted jailbreak probability with 1 to determine the prediction accuracy, updating the parameters of the image-text pair based on the prediction accuracy, until the corresponding prediction accuracy reaches the accuracy threshold, ending the update, and obtaining the adversarial example pair. For example, if the accuracy threshold is set to 80%, then when the prediction accuracy reaches 0.8, it is considered that the update can end. The specific settings can be flexibly configured according to actual needs.

[0095] In some possible implementations, the hidden state is input into a probability prediction model in the above steps to obtain a predicted jailbreak probability, including:

[0096] Each hidden state is input into the corresponding probability prediction model to obtain the corresponding predicted jailbreak probability.

[0097] The target large model includes multiple transformation modules, each of which outputs a hidden state. There are multiple probability prediction models, and each probability prediction model corresponds to a transformation module.

[0098] There can be a sequential relationship between the various conversion modules, that is, the input of one conversion module is the output of the next conversion module.

[0099] Specifically, the pre-trained probability prediction model is connected to the corresponding conversion module. Then, the test data is input into the target multimodal large model, and the hidden states output by each conversion module of its text processing part are obtained. After obtaining the hidden states, these hidden states are input into the probability prediction model at the corresponding position to obtain multiple predicted jailbreak probabilities.

[0100] In the specific implementation process, the intermediate hidden states corresponding to each conversion module are obtained. Each intermediate hidden state can be input into the corresponding probability prediction model, or a target hidden state can be generated based on multiple intermediate hidden states. The target hidden state is then input into the probability prediction model to obtain the predicted jailbreak probability. The probability prediction model used here is also trained based on the hidden state composed of multiple intermediate states. The specific method for obtaining the predicted jailbreak probability can be selected according to actual needs.

[0101] In some possible implementations, the above steps involve updating the image-text pair based on the predicted jailbreak probability until a first predetermined termination condition is met, at which point the update ends, resulting in adversarial example pairs, including:

[0102] Based on each predicted jailbreak probability, update the image-text pair until the first predetermined termination condition is met, at which point the update ends and an adversarial sample pair is obtained.

[0103] Specifically, the predicted jailbreak probability corresponding to each hidden state is obtained, and each predicted jailbreak probability is made higher and higher, so that the predicted jailbreak probability gets closer and closer to 1. The image-text pair is updated until the first predetermined termination condition is reached, the update ends, and the adversarial sample pair is obtained.

[0104] In the specific implementation process, in order to generate adversarial examples with jailbreak capabilities, the input image is updated in the direction of maximizing the predicted jailbreak probability, that is, minimizing the difference between the jailbreak probability predicted by each probability prediction model and 1 (1 represents a 100% probability of jailbreak) to obtain the gradient. After obtaining the gradient, the input image is iteratively updated using the gradient to gradually increase its jailbreak probability. The loss function can be the mean squared error loss function.

[0105] In some possible implementations, the above steps update the image-text pairs based on each predicted jailbreak probability until a first predetermined termination condition is met, at which point the update ends, resulting in adversarial example pairs, including:

[0106] Based on the predicted jailbreak probability and target probability, determine the mean squared error loss function;

[0107] Based on the mean squared error loss function, the parameters of the image-text pair are updated until the first predetermined termination condition is met, at which point the update ends and an adversarial example pair is obtained.

[0108] The target probability can be set based on actual needs.

[0109] Specifically, based on the predicted escape probability and target probability corresponding to each hidden state, the mean squared error loss function is determined. Based on the mean squared error loss function, the parameters of the image-text pair are updated until the first predetermined termination condition is reached, at which point the update ends, and the adversarial sample pair is obtained.

[0110] In the specific implementation process, a probabilistic prediction model is used to perform jailbreak probability optimization (JPO). This enhances the jailbreak capability of the input by maximizing the predicted jailbreak probability of the input's hidden states. The jailbreak probability is maximized by minimizing a loss function, which can be:

[0111]

[0112] Among them, L JPO It is a loss function. This predicts the probability of jailbreak, where 1 represents a 100% probability of jailbreak, and L represents the probability of jailbreak. MSE This is the formula used to calculate the mean squared error loss function.

[0113] Specifically, the gradient of the loss function with respect to the image-text pair is calculated, with the gradient direction being the direction that minimizes the loss function. This gradient is then used to update the image in the image-text pair.

[0114]

[0115] Where, x′ img As image parameters, x′ can be... img Initialize to x imh α is the single-step step size, sign is the sign function, and x′ img L JPO Let x′ be the gradient of the loss function with respect to the input. After multiple iterations, x′ img The hidden state will get closer and closer to the hidden state with a high jailbreak probability, thus enhancing its jailbreak capability. Compared with existing methods, the above jailbreak probability optimization method improves the jailbreak capability of the input by maximizing the jailbreak probability of the hidden state, avoiding direct conflict with security alignment. Therefore, it does not require too many iterations and a large perturbation range. In addition, the above jailbreak probability optimization method only uses a single probability prediction model to perform a single-digit regression task, which is faster and easier to converge than existing methods.

[0116] In practical implementation, the formula for the mean squared error loss function is as follows:

[0117]

[0118] Where y is the true value, corresponding to the target probability. is the predicted value, corresponding to the predicted jailbreak probability, n is the number of samples, corresponding to the number of hidden states, and i is the index of the sample.

[0119] In some possible implementations, the above steps involve updating the image-text pair based on the predicted jailbreak probability until a first predetermined termination condition is met, at which point the update ends, resulting in adversarial example pairs, including:

[0120] Determine the jailbreak loss function based on the predicted jailbreak probability;

[0121] The parameters of the image-text pair are updated based on the jailbreak loss function until the first predetermined termination condition is met, at which point the update ends and an adversarial sample pair is obtained.

[0122] Specifically, based on the predicted jailbreak probability, a jailbreak loss function is determined, and the parameters of the image-text pair are updated based on the jailbreak loss function until the first predetermined termination condition is met, at which point the update ends, and adversarial sample pairs are obtained.

[0123] In the specific implementation process, the first predetermined termination condition may include the predicted probability reaching a preset threshold, performing gradient updates on the predicted jailbreak probability to update the parameters of the image-text pair until the predicted probability reaches the preset threshold, ending the training and obtaining the probabilistic prediction model; the first predetermined termination condition may also include the jailbreak loss function converging, determining the jailbreak loss function based on the predicted jailbreak probability, updating the parameters of the image-text pair based on the jailbreak loss function until the jailbreak loss function converges, ending the training and obtaining the trained probabilistic prediction model; the first predetermined termination condition may also include the probability difference being less than or equal to a preset difference threshold, using the absolute value of the difference between the predicted jailbreak probability and the reference jailbreak probability as the probability difference, updating the parameters of the initial probabilistic model based on the probability difference until the probability difference is less than or equal to the preset difference threshold, ending the training and obtaining the trained probabilistic prediction model.

[0124] In some possible implementations, the above steps involve updating the parameters of the image-text pair based on a jailbreak loss function until a first predetermined termination condition is met, at which point the update ends, resulting in an adversarial example pair, including:

[0125] The parameters of the image-text pair are updated based on the jailbreak loss function, and the number of iterations for the image-text pair is determined.

[0126] If the number of iterations reaches the first predetermined termination condition, the update ends, and the adversarial sample pair is obtained.

[0127] Specifically, the parameters of the image-text pair are updated based on the jailbreak loss function. Since the response result is uncertain, the jailbreak loss function may not converge. Therefore, the number of iterations for the image-text pair can be determined. If the number of iterations reaches the first predetermined termination condition, the update ends and the adversarial sample pair is obtained. In actual use, the number of iterations can be set to 150 or 200 rounds, which is much lower than the existing technology.

[0128] In the specific implementation process, the number of iterations required to generate adversarial examples can be preset. The number of iterations is recorded when updating the image-text pair. The parameters of the image-text pair are updated based on the jailbreak loss function. When the number of iterations reaches the preset number, the update stops, and the adversarial example pair is obtained. Existing technologies may require 5,000 rounds to generate a general adversarial example, while this solution only requires 200 rounds, thus reducing time costs. Existing methods do not limit the perturbation range, while the perturbation range of this method is very small, resulting in higher invisibility. This is due to avoiding direct conflicts with security alignment. Therefore, this solution is superior to existing technologies in terms of attack success rate, number of attack iteration rounds, and perturbation range.

[0129] In some possible implementations, such as Figure 4 As shown, the above method also includes:

[0130] S410, Obtain the training sample set.

[0131] The training sample set includes multiple training samples.

[0132] Specifically, a training text set can be obtained from a pre-defined adversarial dataset. Each text in the training text set is combined with an image to obtain multiple training samples, which can be a training sample set. The training sample can be a training image-text pair consisting of a default image and a text.

[0133] In the specific implementation process, the training sample set can be obtained by combining the training text set and a default image. The training text set can be obtained by dividing the adversarial dataset. The adversarial dataset can include the adversarial example evaluation dataset AdvBench, the attack rejection dataset StrongREJECT, and the jailbreak adversarial dataset JailbreakBench. The adversarial dataset can be divided into training text set and test text set according to the ratio. For example, the adversarial dataset can be divided into training text set and test text set according to the ratio of 0.8 and 0.2. The training text set is used to train the probabilistic prediction model, and the test text set can be used to generate adversarial examples.

[0134] S420 inputs the training samples one by one into the target large model to obtain multiple corresponding response results.

[0135] One training sample corresponds to multiple response results, and the number of response results can be preset.

[0136] Specifically, the training samples are input into the target large model one by one to obtain multiple response results for each training sample. The images and text in the training samples are processed and analyzed to obtain the corresponding response results. For a training sample, there can be multiple different and reasonable responses. The number of responses can be set based on actual needs. For example, considering both accuracy and time cost, 20 responses can be generated for each training sample.

[0137] In the specific implementation process, the training samples can include images and text. The training samples are input into the target large model. The images in the training samples are input into the image processing module of the target large model, and the text in the training samples are input into the text processing module of the target large model. After extracting the features of the corresponding images and corresponding texts respectively, the image features are mapped to the text space using a connector and concatenated with the text features. The text units are processed one by one using an autoregressive method to generate the response results.

[0138] S430 determines the corresponding reference jailbreak probability based on multiple response results.

[0139] Specifically, for each response, a preset judgment rule is used to assess the harmfulness of the response content, and the assessment result corresponding to each response is determined. Based on each response result of the same training sample, the corresponding reference jailbreak probability is determined. Here, harmfulness can be used to indicate whether the corresponding response result can enable the large model to jailbreak.

[0140] In the specific implementation process, the preset judgment rules may include comparing the response results with the harmful dataset and determining the harmfulness assessment result based on the similarity obtained from the comparison; or inputting the response results into the harmful text assessment model to obtain the assessment result; or obtaining the assessment result based on manual selection; any specific method that can determine whether the response results can enable the large model to jailbreak can be used, and the specific method can be selected according to actual needs without limitation.

[0141] S440: Obtain the hidden state of the target large model for the training samples, input the hidden state into the initial prediction model, and obtain the predicted jailbreak probability.

[0142] The initial prediction model can be a multilayer perceptron (MLP) model, which contains at least three layers: an input layer, one or more hidden layers, and an output layer.

[0143] Specifically, the hidden state of the target large model when processing training samples is obtained, the hidden state is used as the input of the initial prediction model, the reference jailbreak probability is used as the label, and the predicted jailbreak probability output by the initial prediction model is obtained. Based on the reference jailbreak probability, the initial prediction model is trained and the model parameters are modified so that the predicted jailbreak probability gets closer and closer to the reference jailbreak probability. The relationship between the hidden state and the jailbreak probability is constructed through the model, which helps to generate adversarial examples using the jailbreak probability.

[0144] In the specific implementation process, the target large model can include multiple transformation modules. Each transformation module can output a corresponding hidden state. Obtaining the hidden state of the target large model is also obtaining the hidden state of each module. Each transformation module is set with a corresponding initial prediction model. The hidden state of each transformation module is input into the corresponding initial prediction model to obtain their respective predicted jailbreak probabilities. The initial model parameters are updated based on the predicted jailbreak probabilities and the reference jailbreak probabilities.

[0145] S450 updates the parameters of the initial probability model based on the reference jailbreak probability and the predicted jailbreak probability until the second predetermined termination condition is met, thus ending the training and obtaining the trained probability prediction model.

[0146] The second predetermined termination condition may include: accuracy, recall or loss function value reaching a threshold, training time reaching a maximum value, the gradient change of the model being very small, and performance starting to decline.

[0147] Specifically, the probability of jailbreak can be compared with the predicted probability of jailbreak to update the parameters of the initial probability model, so that the model reaches the second predetermined termination condition, ends the training, and obtains the trained probability prediction model.

[0148] In the specific implementation process, the second predetermined termination condition includes the convergence of the probability loss function. Based on the reference jailbreak probability and the predicted jailbreak probability, the probability loss function is determined. The parameters of the initial probability model are updated based on the probability loss function until the probability loss function converges, at which point training ends, and a trained probability prediction model is obtained. The second predetermined termination condition may also include the probability difference being less than or equal to a preset difference threshold. The absolute value of the difference between the predicted jailbreak probability and the reference jailbreak probability is used as the probability difference. The parameters of the initial probability model are updated based on the probability difference until the probability difference is less than or equal to the preset difference threshold, at which point training ends, and a trained probability prediction model is obtained.

[0149] In the specific implementation process, the second predetermined termination condition may also include the accuracy reaching a preset threshold, comparing the reference jailbreak probability with the predicted jailbreak probability, determining the accuracy of the predicted jailbreak probability, updating the parameters of the large model to be trained based on the accuracy, until the accuracy reaches the preset threshold, ending the training, and obtaining the probability prediction model.

[0150] In practical implementation, the probability prediction model can also be in the form of a network. To improve the jailbreak capability of the input, a Jailbreak Probability Prediction Network (JPPN) can be constructed and trained. JPPN is used to model the relationship between the hidden state of the input and the jailbreak probability. Specifically, a three-layer MLP model can be used to construct JPPN. The input of JPPN is the hidden state of the large model, and the output of JPPN is the prediction of the jailbreak probability of the hidden state of the input, which is a real number greater than or equal to 0 and less than or equal to 1. Since each transformation module in the target multimodal large model outputs an intermediate state, this invention initializes a separate JPPN for each transformation module and connects it to the output position of that transformation module. The training process of JPPN can use the mean squared error loss function, and the optimization objective is to minimize the difference between the approximate jailbreak probability of an image-text pair and the jailbreak probability of the hidden state of this image-text pair predicted by JPPN. In practical applications, the training rounds of the probabilistic prediction model can be set to 150 rounds, and parameter updates can be stopped once the training rounds are reached to obtain the probabilistic prediction network. Alternatively, the initial learning rate can be set to 0.001, and the learning rate can be reduced to 20% of the original rate every 50 rounds.

[0151] In some possible implementations, the above steps determine the corresponding reference jailbreak probability based on multiple response results, including:

[0152] Based on the preset content evaluation components, determine the content evaluation result of each response;

[0153] Based on the content evaluation results, the corresponding reference jailbreak probability is determined.

[0154] The content assessment results include harmful and harmless categories.

[0155] Specifically, based on a preset content evaluation component, the content of the response results is evaluated to determine whether the response results are harmful content, and the content evaluation result of each response result is obtained. Based on whether the content evaluation result is harmful or harmless, an approximate jailbreak probability is calculated, and the approximate jailbreak probability is used as a reference jailbreak probability.

[0156] In practice, the content evaluation component can score each reply, assigning a score of 0 to replies containing harmful content, a score of 1 to rejection or irrelevant content, and a score of 2 to positive guidance content. A score of 0 can be considered a successful attack, while scores of 1 and 2 can be considered failed attacks. By obtaining the content evaluation results of multiple replies for an input, the approximate jailbreak probability of that input can be calculated, which is the aforementioned reference jailbreak probability. This approximate jailbreak probability is defined as the ratio of the number of successful jailbreaks to the number of replies in the multiple replies corresponding to this input.

[0157] In some possible implementations, the steps described above, based on the content evaluation results, determine the corresponding reference jailbreak probability, including:

[0158] The number of responses deemed harmful by the content assessment will be used as the first number;

[0159] The reference jailbreak probability is determined based on the number of response results and the first number.

[0160] The content assessment results include harmful and harmless categories.

[0161] Specifically, for a training sample, determine the total number of corresponding response results, determine the first number of response results whose content evaluation result is harmful, and determine the corresponding reference jailbreak probability based on the relationship between the first number and the total number.

[0162] In practice, the same input may result in a successful jailbreak or a failed jailbreak. The jailbreak probability can be defined by dividing the number of successful jailbreaks by the total number of queries. Specifically, the jailbreak probability of X on M is... Defined as:

[0163]

[0164] Where N is the number of responses, M represents the multimodal target large model, and X represents the training samples input to the target large model, specifically X = (x img ,x txt ), x img For the images in the training samples, x txt C represents the text in the training samples, and C is the criterion for judging whether the model output is harmful, which can be achieved through a preset harmfulness judgment interface.

[0165] Furthermore, a limited number of responses can be generated, and the percentage of responses that result in a successful jailbreak can be calculated to obtain the jailbreak probability. An approximation of this value is used as a reference for the jailbreak probability. This approximation can be defined as the approximate jailbreak probability.

[0166]

[0167] in, To approximate the probability of escape, n is the number of responses in a finite number of attempts, M represents the large multimodal target model, and X = (x img ,x txt ) represents the training samples of the input target large model, x img For the images in the training samples, x txtLet C be the text in the training samples, and C be the criterion for judging whether the model output is harmful. According to the law of large numbers, as the number of responses used to calculate the approximate probability of jailbreak gradually increases, It will gradually converge to

[0168] In some possible implementations, the above steps involve updating the parameters of the initial probability model based on the reference jailbreak probability and the predicted jailbreak probability until a second predetermined termination condition is met, thus ending the training and obtaining a trained probability prediction model, including:

[0169] The absolute value of the difference between the predicted jailbreak probability and the reference jailbreak probability is used as the probability difference.

[0170] The parameters of the initial probability model are updated based on the probability difference until the probability difference is less than or equal to a preset difference threshold, at which point the training ends and a well-trained probability prediction model is obtained.

[0171] The second predetermined termination condition includes the probability difference being less than or equal to a preset difference threshold.

[0172] Specifically, the absolute value of the difference between the predicted jailbreak probability and the reference jailbreak probability is used as the probability difference. The probability difference is reduced, and the parameters of the initial probability model are updated according to the probability difference until the probability difference is less than or equal to the preset difference threshold. The training ends, and a probability prediction model that can determine the likelihood of jailbreak in a hidden state is obtained.

[0173] In the specific implementation process, in order to ensure that the probability prediction model can accurately predict the jailbreak probability of different inputs after training, it is necessary to test the accuracy of the probability prediction model. Since predicting the jailbreak probability is a regression problem, and the true value of this regression problem has a certain degree of randomness (this is because the reference jailbreak probability depends on the response content generated by the multimodal target large model, and the response content is random), this application proposes a τ threshold accuracy index: when the absolute value of the difference between the predicted jailbreak probability and the true value is less than or equal to τ, the prediction is considered accurate; otherwise, it is considered inaccurate. The τ threshold accuracy is calculated by dividing the number of accurate predictions by the total number of predictions. For example, τ can be 0.2 or 0.3, which can be calculated according to the actual situation.

[0174] In the above embodiments, by cyclically inputting image-text pairs into the target large model, the corresponding hidden states are obtained. The hidden states are then input into the probability prediction model to obtain the predicted jailbreak probability. Based on the predicted jailbreak probability, the image-text pairs are updated until the termination condition is met, resulting in adversarial example pairs. By associating the hidden states with the jailbreak probability and using the predicted jailbreak probability to guide the updating of the image-text pairs, adversarial examples with stronger jailbreak capabilities can be obtained. This eliminates the need to counteract the influence of security alignment, reduces the number of iterations for generating adversarial examples, shortens the time, and improves the generation efficiency of adversarial example pairs.

[0175] Furthermore, based on the output of the large model, the reference jailbreak probability of the training samples is determined, the corresponding hidden state is obtained, and the hidden state is input into the initial prediction model to obtain the predicted jailbreak probability. Based on the reference jailbreak probability and the predicted jailbreak probability, the parameters of the initial probability model are updated until the predetermined termination condition is reached, and the trained probability prediction model is obtained. By constructing the relationship between the hidden state and the jailbreak probability through the probability prediction model, the image-text pair can be updated quickly based on the predicted jailbreak probability, which effectively improves the generation efficiency of adversarial example pairs.

[0176] In addition, the target large model can include multiple transformation modules, each of which outputs a hidden state and a probability prediction model. The hidden state is input into the corresponding probability prediction model to obtain the predicted jailbreak probability corresponding to the hidden state. The mean squared error loss function is determined by multiple predicted jailbreak probabilities, thereby updating the image-text pair. The hidden states of each transformation module are fully considered, making the updated adversarial example pair have a stronger jailbreak capability.

[0177] In one example, the large model adversarial example pair generation method of this application, such as Figure 5 As shown, it may include:

[0178] The image and text pairs are input into the target large model to obtain the corresponding hidden states. The target large model includes multiple transformation modules, each of which outputs a hidden state. There are multiple probability prediction models, and each probability prediction model corresponds to a transformation module.

[0179] Each hidden state is input into the corresponding probability prediction model to obtain the corresponding predicted jailbreak probability;

[0180] Based on the predicted jailbreak probability and target probability, determine the mean squared error loss function;

[0181] Based on the mean squared error loss function, the parameters of the image-text pair are updated to determine whether the first predetermined termination condition has been met.

[0182] If the first predetermined termination condition is met, the update ends and an adversarial sample pair is obtained;

[0183] If the first predetermined termination condition is not met, the image-text pair will continue to be updated until the first predetermined termination condition is met, at which point the update will end and an adversarial sample pair will be obtained.

[0184] The aforementioned large-scale adversarial example pair generation method obtains the corresponding hidden states by iteratively inputting image-text pairs into the target large-scale model. The hidden states are then input into a probability prediction model to obtain the predicted jailbreak probability. Based on the predicted jailbreak probability, the image-text pairs are updated until a termination condition is met, resulting in adversarial example pairs. By associating the hidden states with the jailbreak probability and using the predicted jailbreak probability to guide the updating of the image-text pairs, adversarial examples with stronger jailbreak capabilities can be obtained. This method does not require offsetting the influence of security alignment, reduces the number of iterations in adversarial example generation, shortens the time, and improves the generation efficiency of adversarial example pairs.

[0185] Furthermore, based on the output of the large model, the reference jailbreak probability of the training samples is determined, the corresponding hidden state is obtained, and the hidden state is input into the initial prediction model to obtain the predicted jailbreak probability. Based on the reference jailbreak probability and the predicted jailbreak probability, the parameters of the initial probability model are updated until the predetermined termination condition is reached, and the trained probability prediction model is obtained. By constructing the relationship between the hidden state and the jailbreak probability through the probability prediction model, the image-text pair can be updated quickly based on the predicted jailbreak probability, which effectively improves the generation efficiency of adversarial example pairs.

[0186] In addition, the target large model can include multiple transformation modules, each of which outputs a hidden state and a probability prediction model. The hidden state is input into the corresponding probability prediction model to obtain the predicted jailbreak probability corresponding to the hidden state. The mean squared error loss function is determined by multiple predicted jailbreak probabilities, thereby updating the image-text pair. The hidden states of each transformation module are fully considered, making the updated adversarial example pair have a stronger jailbreak capability.

[0187] This application provides a large model adversarial example pair generation device, such as... Figure 6 As shown, the large model adversarial example pair generation device 60 may include: a state acquisition module 610, a probability prediction module 620, and a sample generation module 630, wherein,

[0188] The state acquisition module 610 is used to cyclically input image-text pairs into the target large model to obtain the corresponding hidden states;

[0189] The probability prediction module 620 is used to input each hidden state into the probability prediction model to obtain the predicted jailbreak probability.

[0190] The sample generation module 630 is used to update the image-text pair based on the predicted jailbreak probability until a first predetermined termination condition is met, at which point the update ends and an adversarial sample pair is obtained; the adversarial sample pair is the updated image-text pair.

[0191] Specifically, the sample generation module 630 may include a state acquisition module 610 and a probability prediction module 620. The state acquisition module 610 and the probability prediction module 620 can be directly used to update image-text pairs when concatenated.

[0192] As an optional embodiment, in this device, the probability prediction module 620 is specifically used for:

[0193] Each hidden state is input into the corresponding probability prediction model to obtain the corresponding predicted jailbreak probability.

[0194] As an optional embodiment, in this device, the sample generation module 630 is specifically used for:

[0195] Based on each predicted jailbreak probability, update the image-text pair until the first predetermined termination condition is met, at which point the update ends and an adversarial sample pair is obtained.

[0196] As an optional embodiment, in this device, the sample generation module 630 is specifically used for:

[0197] Based on the predicted jailbreak probability and target probability, determine the mean squared error loss function;

[0198] Based on the mean squared error loss function, the parameters of the image-text pair are updated until the first predetermined termination condition is met, at which point the update ends and an adversarial example pair is obtained.

[0199] As an optional embodiment, in this device, the sample generation module 630 is specifically used for:

[0200] Determine the jailbreak loss function based on the predicted jailbreak probability;

[0201] The parameters of the image-text pair are updated based on the jailbreak loss function until the first predetermined termination condition is met, at which point the update ends and an adversarial sample pair is obtained.

[0202] As an optional embodiment, in this device, the sample generation module 630 is specifically used for:

[0203] The parameters of the image-text pair are updated based on the jailbreak loss function, and the number of iterations for the image-text pair is determined.

[0204] If the number of iterations reaches the first predetermined termination condition, the update ends, and the adversarial sample pair is obtained.

[0205] As an optional embodiment, the device further includes a probability model training module, specifically used for:

[0206] Obtain the training sample set; the training sample set includes multiple training samples.

[0207] The training samples are input one by one into the target large model to obtain multiple corresponding response results;

[0208] Based on multiple responses, determine the corresponding reference jailbreak probability;

[0209] Obtain the hidden state of the target large model for the training samples, input the hidden state into the initial prediction model, and obtain the predicted jailbreak probability.

[0210] Based on the reference jailbreak probability and the predicted jailbreak probability, the parameters of the initial probability model are updated until the second predetermined termination condition is met, at which point the training ends and a well-trained probability prediction model is obtained.

[0211] As an optional embodiment, the probability model training module in this device is specifically used for:

[0212] Based on the preset content evaluation components, determine the content evaluation result of each response;

[0213] Based on the content evaluation results, the corresponding reference jailbreak probability is determined.

[0214] As an optional embodiment, the probability model training module in this device is specifically used for:

[0215] The number of responses deemed harmful by the content assessment will be used as the first number;

[0216] The reference jailbreak probability is determined based on the number of response results and the first number.

[0217] As an optional embodiment, the probability model training module in this device is specifically used for:

[0218] The absolute value of the difference between the predicted jailbreak probability and the reference jailbreak probability is used as the probability difference.

[0219] The parameters of the initial probability model are updated based on the probability difference until the probability difference is less than or equal to a preset difference threshold, at which point the training ends and a well-trained probability prediction model is obtained.

[0220] The large-scale adversarial example pair generation device provided in this application obtains the corresponding hidden states by cyclically inputting image-text pairs into a target large-scale model. The hidden states are then input into a probability prediction model to obtain the predicted jailbreak probability. Based on the predicted jailbreak probability, the image-text pairs are updated until a termination condition is met, resulting in adversarial example pairs. By associating the hidden states with the jailbreak probability and using the predicted jailbreak probability to guide the updating of the image-text pairs, adversarial examples with stronger jailbreak capabilities can be obtained. This eliminates the need to counteract the effects of security alignment, reduces the number of iterations in adversarial example generation, shortens the time, and improves the generation efficiency of adversarial example pairs.

[0221] Furthermore, based on the output of the large model, the reference jailbreak probability of the training samples is determined, the corresponding hidden state is obtained, and the hidden state is input into the initial prediction model to obtain the predicted jailbreak probability. Based on the reference jailbreak probability and the predicted jailbreak probability, the parameters of the initial probability model are updated until the predetermined termination condition is reached, and the trained probability prediction model is obtained. By constructing the relationship between the hidden state and the jailbreak probability through the probability prediction model, the image-text pair can be updated quickly based on the predicted jailbreak probability, which effectively improves the generation efficiency of adversarial example pairs.

[0222] In addition, the target large model can include multiple transformation modules, each of which outputs a hidden state and a probability prediction model. The hidden state is input into the corresponding probability prediction model to obtain the predicted jailbreak probability corresponding to the hidden state. The mean squared error loss function is determined by multiple predicted jailbreak probabilities, thereby updating the image-text pair. The hidden states of each transformation module are fully considered, making the updated adversarial example pair have a stronger jailbreak capability.

[0223] The apparatus in this application embodiment can execute the method provided in this application embodiment, and its implementation principle is similar, and it has corresponding technical effects. The actions performed by each module in the apparatus of each embodiment of this application correspond to the steps in the method of each embodiment of this application. For a detailed functional description of each module of the apparatus, please refer to the description in the corresponding method shown above, which will not be repeated here.

[0224] This application provides an electronic device including a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to implement the steps of the method provided in any optional embodiment of this application. Compared with the prior art, it can achieve the following: by cyclically inputting image-text pairs into a target large model to obtain corresponding hidden states, inputting the hidden states into a probability prediction model to obtain predicted jailbreak probabilities, updating image-text pairs based on the predicted jailbreak probabilities, until a termination condition is reached to obtain adversarial example pairs, associating the hidden states with the jailbreak probabilities, and using the predicted jailbreak probabilities to guide the updating of image-text pairs, thereby obtaining adversarial examples with stronger jailbreak capabilities. It does not require offsetting the influence of security alignment, can reduce the number of iterations for adversarial example generation, shorten the time, and improve the generation efficiency of adversarial example pairs.

[0225] In one alternative embodiment, an electronic device is provided, such as Figure 7 As shown, Figure 7 The illustrated electronic device 7000 includes a processor 7001 and a memory 7003. The processor 7001 and the memory 7003 are connected, for example, via a bus 7002. Optionally, the electronic device 7000 may further include a transceiver 7004, which can be used for data interaction between the electronic device and other electronic devices, such as sending and / or receiving data. It should be noted that in practical applications, the transceiver 7004 is not limited to one type, and the structure of the electronic device 7000 does not constitute a limitation on the embodiments of this application.

[0226] Processor 7001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 7001 may also be a combination that implements computing functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0227] Bus 7002 may include a pathway for transmitting information between the aforementioned components. Bus 7002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 7002 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 7 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0228] The memory 7003 may be ROM (Read Only Memory) or other types of static storage devices capable of storing static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices capable of storing information and instructions, or EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media, other magnetic storage devices, or any other medium capable of carrying or storing computer programs and capable of being read by a computer, without limitation herein.

[0229] The memory 7003 is used to store computer programs that execute the embodiments of this application, and its execution is controlled by the processor 7001. The processor 7001 is used to execute the computer programs stored in the memory 7003 to implement the steps shown in the foregoing method embodiments.

[0230] Among them, electronic devices include, but are not limited to, servers, terminals or components capable of implementing the above-mentioned adversarial sample pair generation method.

[0231] This application provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it can implement the steps and corresponding content of the aforementioned method embodiments.

[0232] It should be noted that the computer-readable storage medium described above in this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0233] This application also provides a computer program product, including a computer program that, when executed by a processor, can implement the steps and corresponding content of the aforementioned method embodiments.

[0234] The terms "first," "second," "third," "fourth," "1," "2," etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in a sequence other than that shown in the figures or text.

[0235] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0236] It should be understood that although arrows indicate various operation steps in the flowcharts of this application's embodiments, the order in which these steps are implemented is not limited to the order indicated by the arrows. Unless explicitly stated herein, in some implementation scenarios of this application's embodiments, the implementation steps in each flowchart can be executed in other orders as required. Furthermore, some or all steps in each flowchart, based on the actual implementation scenario, may include multiple sub-steps or multiple stages. Some or all of these sub-steps or stages can be executed at the same time, and each sub-step or stage can also be executed at different times. In scenarios where execution times differ, the execution order of these sub-steps or stages can be flexibly configured according to requirements, and this application's embodiments do not limit this.

[0237] The above description is only an optional implementation method for some implementation scenarios of this application. It should be noted that for those skilled in the art, other similar implementation methods based on the technical concept of this application without departing from the technical concept of this application also fall within the protection scope of the embodiments of this application.

Claims

1. A large model adversarial sample pair generation method, characterized in that, The method comprises the following steps: inputting the image-text pair into a target large model to obtain a corresponding hidden state; inputting the hidden state into a probability prediction model to obtain a predicted jailbreak probability; updating the image-text pair based on the predicted jailbreak probability until a first predetermined end condition is reached, ending the update to obtain an adversarial sample pair; the adversarial sample pair is an updated image-text pair; the target large model comprises a plurality of conversion modules, each conversion module corresponds to an output hidden state, and the probability prediction model has a plurality of probability prediction models, each probability prediction model corresponds to a conversion module; the method comprises the following steps: inputting each hidden state into the corresponding probability prediction model to obtain a corresponding predicted jailbreak probability; the method comprises the following steps: based on the predicted jailbreak probability and the target probability, determine the mean square error loss function; calculate the gradient of the mean square error loss function with respect to the image-text pair; update the image in the image-text pair using the gradient.

2. The method of claim 1, wherein, the method comprises the following steps: based on each predicted jailbreak probability, update the image-text pair until the first predetermined end condition is reached, end the update to obtain the adversarial sample pair.

3. The method of claim 2, wherein, the method comprises the following steps: based on each predicted jailbreak probability and the target probability, determine the mean square error loss function; based on the mean square error loss function, update the parameters of the image-text pair until the first predetermined end condition is reached, end the update to obtain the adversarial sample pair.

4. The method of claim 1, wherein, the method comprises the following steps: based on the predicted jailbreak probability, determine the jailbreak loss function; based on the jailbreak loss function, update the parameters of the image-text pair until the first predetermined end condition is reached, end the update to obtain the adversarial sample pair.

5. The method of claim 4, wherein, the method comprises the following steps: based on the jailbreak loss function, update the parameters of the image-text pair, and determine the iteration number of the image-text pair; if the iteration number reaches the first predetermined end condition, end the update to obtain the adversarial sample pair.

6. The method of claim 1, wherein, the method further comprises the following steps: obtain a training sample set; the training sample set comprises a plurality of training samples; input the training samples into the target large model one by one to obtain a plurality of corresponding reply results; based on the plurality of reply results, determine the corresponding reference jailbreak probability; obtain the hidden state of the target large model for the training sample, input the hidden state into an initial prediction model to obtain a predicted jailbreak probability; Based on the reference jailbreaking probability and the predicted jailbreaking probability, the initial probability model is updated in parameters until a second predetermined end condition is reached, ending training, to obtain the trained probability prediction model.

7. The method of claim 6, wherein, The reference jailbreaking probability corresponding to each of the multiple reply results is determined based on the multiple reply results. Based on a preset content evaluation component, a content evaluation result of each of the reply results is determined. Based on the content evaluation result, the reference jailbreaking probability corresponding to each of the reply results is determined.

8. The method of claim 7, wherein, The content evaluation result includes harmfulness. Based on the content evaluation result, the reference jailbreaking probability corresponding to each of the reply results is determined. The number of reply results with the content evaluation result being harmfulness is taken as a first number. Based on the number of reply results and the first number, the reference jailbreaking probability is determined.

9. The method of claim 6, wherein, The second predetermined end condition includes a probability difference value being less than or equal to a preset difference threshold value. Based on the reference jailbreaking probability and the predicted jailbreaking probability, the initial probability model is updated in parameters until a second predetermined end condition is reached, ending training, to obtain the trained probability prediction model, including: The absolute value of the difference between the predicted jailbreaking probability and the reference jailbreaking probability is taken as the probability difference value. Based on the probability difference value, the initial probability model is updated in parameters until the probability difference value is less than or equal to the preset difference threshold value, ending training, to obtain the trained probability prediction model.

10. A large model adversarial sample pair generation device, characterized in that, Including: A state acquisition module configured to input an image-text pair into a target large model to obtain corresponding hidden states; A probability prediction module configured to input each of the hidden states into a probability prediction model to obtain a predicted jailbreaking probability; A sample generation module configured to update the image-text pair based on the predicted jailbreaking probability until a first predetermined end condition is reached, ending the update, to obtain an adversarial sample pair; the adversarial sample pair is an updated image-text pair. The target large model includes multiple conversion modules, each of which corresponds to outputting a hidden state, and the probability prediction model has multiple probability prediction models, which correspond one-to-one to the conversion modules. The probability prediction module is specifically configured to: Input each of the hidden states into a corresponding probability prediction model to obtain a corresponding predicted jailbreaking probability. The sample generation module is specifically configured to: Determine a mean square error loss function based on the predicted jailbreaking probability and a target probability; Calculate the gradient of the mean square error loss function with respect to the image-text pair; Update the image in the image-text pair using the gradient.

11. An electronic device comprising a memory, a processor, and a computer program stored on the memory, wherein the computer program, when executed by the processor, is arranged to perform the method of any one of claims 1 to 10. The processor executes the computer program to implement the steps of the large model adversarial sample pair generation method of any one of claims 1-9.

12. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the large model adversarial sample pair generation method of any one of claims 1-9.

13. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the large model adversarial sample pair generation method of any one of claims 1-9.

Images