Figure graph security improvement method and device based on confrontation prompt mining

Abstract

The invention discloses a method and device for improving security of a text graph model based on adversarial prompt mining, and the method comprises the steps: predefining a plurality of optimizable noise embedding as an optimization target, obtaining an agent embedding closest to the noise embedding in an embedding space through employing a nearest neighbor search method, splicing the agent embedding to the tail part of a clean embedding, and carrying out the optimization of the optimization target; confrontation embedding is obtained, and a text encoder in the CLIP model is used for encoding the confrontation embedding to extract confrontation prompt features; using an image-text matching objective function to maximize the semantic similarity between the confrontation prompt feature and the risk target image feature, and establishing an association relationship between the confrontation prompt and the risk target image; in the process of reversely optimizing noise embedding by using the objective function, the gradient corresponding to proxy embedding is transmitted to the noise embedding construction optimization process, so that noise embedding continuous optimization based on the gradient is realized; and after the optimization is finished, querying the word which is most matched with the noise embedding in the word search space, and splicing the word to a clean prompt to generate a final confrontation prompt. The device comprises a processor and a memory.

Description

Technical Field

[0001] The present invention relates to the field of text-generated images and information content security, and in particular to a method and device for improving the security of text-generated images based on adversarial prompt mining. Background Art

[0002] In recent years, the Wensheng graph model [1-3] By using input text prompts to generate diverse and high-quality images, it has attracted widespread attention from researchers and community users. Several representative commercial products, such as DALL E 3 [4] 、Midjourney [5] 、Stable Diffusion [6] It has been widely used in various fields such as content generation, artistic creation and marketing. However, the widespread application has also caused people to worry about the abuse and misuse of the cultural graph model. For example, using the cultural graph model to generate some risky and harmful images has a negative impact on the dissemination of information. Therefore, it has become an urgent task to develop safety protection measures for cultural graph models to prevent the generation of harmful images. [7-8] Researchers have also made continuous efforts in this field to improve the security of generated images. For example, DALL·E 3 and Midjourney have deployed image auditors to identify harmful generated images. However, these image auditors are usually trained only on limited real images, making it difficult to effectively detect diverse generated images. [9] .

[0003] To alleviate this problem, Wenshengtu currently relies more on auditing input text prompts to ensure the security of generated images. However, these protection measures still have loopholes. Malicious users can use adversarial prompts (specifically text prompts that do not contain harmful words) to induce images to generate risky images, which seriously threatens the security of Wenshengtu models. [10-11] .

[0004] To this end, it is necessary to study the security vulnerabilities of the text-based graph model and explore how to induce the generation of risky images when the text prompts do not contain the target words, and then use these adversarial prompts to further ensure the security of the model. Summary of the invention

[0005] The present invention provides a method and device for improving the security of a text graph based on adversarial prompt mining. The present invention automatically generates adversarial prompts corresponding to risk targets, and then uses these prompts to achieve stronger protection for the text graph model, as described below:

[0006] In a first aspect, a method for improving the security of a text graph model based on adversarial hint mining comprises:

[0007] Predefine several optimizable noise embeddings as optimization targets, and use the nearest neighbor search method to obtain the proxy embedding closest to the noise embedding in the embedding space, and splice it to the tail of the clean embedding to obtain the adversarial embedding. Use the text encoder in the CLIP model to encode the adversarial embedding to extract the features of the adversarial prompt.

[0008] By using the image-text matching objective function, the semantic similarity between the adversarial prompt features and the risk target image features is maximized, and the association relationship between the adversarial prompt and the risk target image is established;

[0009] In the process of reversely optimizing the noise embedding using the objective function, the gradient corresponding to the proxy embedding is passed to the noise embedding construction optimization process to achieve gradient-based continuous optimization of the noise embedding;

[0010] After the optimization is completed, the word that best matches the noisy embedding is queried in the word search space and concatenated to the clean prompt to generate the final adversarial prompt.

[0011] Wherein, the noise is embedded in E n To: predefine M optimizable embeddings;

[0012]

[0013] in, is the mth embedding in the noise embedding;

[0014] The agent embeds E p To do this: use nearest neighbor search to obtain the text embedding that best matches the noise embedding in the embedding space;

[0015]

[0016] Among them, E s is the text embedding of all words accepted by the CLIP model, is the mth embedding in the noise embedding, The CLIP model can accept text embeddings and Most similar embeddings;

[0017] The adversarial embedding To do this: concatenate the proxy embedding to the end of the clean embedding;

[0018]

[0019] Among them, E x For clean embedding.

[0020] Among them, the image-text matching objective function is:

[0021]

[0022] in, is the feature corresponding to the u-th risk target image, is the adversarial cue feature, and U is the number of risky target images.

[0023] Among them, the gradient corresponding to the proxy embedding is:

[0024]

[0025] Pass the gradient corresponding to the proxy embedding to the noise embedding and perform a gradient update operation:

[0026]

[0027] Among them, η is the learning rate hyperparameter for gradient update.

[0028] The word that best matches the noise embedding is searched in the word search space, and the final adversarial prompt is generated after being concatenated with the clean prompt:

[0029] The first modification deleted all non-English words in the CLIP model word space;

[0030] S={t v |t v ∈Vocab,t v ∈ English}

[0031] The second revision deleted all synonyms related to risk objectives;

[0032] Using a synonym model Word2Vec to obtain the risk target p t And use the word segmenter T in the CLIP model to divide these synonyms into several tokens:

[0033] W pt =T(Word2Vec(p t ))

[0034] Define the word search space S as:

[0035]

[0036] Search the search space for the word that best matches the optimized noisy embedding as the suffix of the clean prompt:

[0037]

[0038] in, is the mth noise embedding learned after optimization, Emb(t m) is the text embedding E that the CLIP model can accept s Zhongyu The most similar embedding, t m Embed the most similar text Embed(t m ) corresponding to the word;

[0039] Append the learned tokens to the clean prompt to obtain the final adversarial prompt:

[0040]

[0041] The method further comprises: training a text classification model, wherein the text classification model comprises: a CLIP text feature encoder It is used to extract the features of the input text prompt, and the classifier fc is used to classify and identify the extracted text features;

[0042] With clean prompt x and learned adversarial prompt As a data benchmark, the parameter update of the text classification model is constrained by the binary classification loss:

[0043]

[0044] Among them, N adv is the number of samples of adversarial prompts, N cln is the number of clean prompt samples, y n is the label corresponding to the nth sample in the dataset, It is the probability value predicted by the text classifier for the nth sample.

[0045] In the second aspect, a device for improving the security of a literary graph model based on adversarial prompt mining is characterized in that the device comprises: a processor and a memory, the memory stores program instructions, and the processor calls the program instructions stored in the memory to enable the device to execute any one of the methods described in the first aspect.

[0046] In a third aspect, a computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor executes any one of the methods described in the first aspect.

[0047] The beneficial effects of the technical solution provided by the present invention are:

[0048] 1. This paper adds a learnable suffix noise embedding to the text embedding to achieve the perturbation of the clean prompt, which can be used by researchers to explore the loopholes of the text graph model in hidden generation of risk content based on the association between the learned suffix words and risk targets;

[0049] 2. The present invention uses an optimization method based on proxy embedding to update noise embedding by using the gradient information of proxy embedding, thereby alleviating the difficulty of discrete text optimization and realizing an efficient gradient-based optimization process of noise embedding;

[0050] 3. The present invention utilizes the image-text matching target loss in the CLIP image-text multimodal model to successfully associate adversarial prompts with risky target images, so that the adversarial prompts can successfully induce the text-based image model to generate risky target images, and then use these adversarial prompts to build a text classification model to filter harmful inputs, enhance the performance of the model's security protection measures, prevent the output of risky images, and improve the security of the network. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 It is a flowchart of a method for improving the security of text graphs based on adversarial hint mining;

[0052] Figure 2 A network structure diagram for improving the security of a text graph based on adversarial hint mining;

[0053] Figure 3 Another network structure diagram for improving the security of a text graph based on adversarial hint mining. DETAILED DESCRIPTION

[0054] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention are described in further detail below.

[0055] In order to solve the technical problems existing in the background technology, the embodiment of the present invention proposes a method for improving the security of the text graph based on adversarial prompt mining, which aims to reveal the security vulnerabilities of the text graph model and enhance the security of the text graph model. The input of this task is a risk target and a clean prompt that is unrelated to the risk target. The goal is to obtain an adversarial prompt by adding a perturbation to the clean prompt, so that the model is induced to generate pictures related to the risk target (hereinafter referred to as risk target pictures) without containing words related to the risk target. Finally, these adversarial prompts are used to achieve safer protection for the text graph model. This task mainly faces the following challenges:

[0056] 1) Strict adversarial prompt generation conditions: This task requires that adversarial prompts must not contain words related to risk targets, and that adversarial prompts must induce the model to generate risk target images. Therefore, how to establish the association between adversarial prompts and risk target images is a challenge;

[0057] 2) Lack of effective perturbation strategy: There is an inverse correlation between the perturbation intensity setting in the clean prompt and the difficulty of generating risky target images. Specifically, in order to simulate the noise conditions that may occur in real situations, it is hoped that the perturbation of the clean prompt is as small as possible, but this small perturbation makes it more difficult to generate risky target images. Therefore, how to design an effective perturbation strategy that takes into account the difficulty of generating risky target images and minimizes the perturbation intensity of the clean prompt is a difficult problem;

[0058] 3) Lack of effective optimization strategies: Text perturbations are usually discrete character or word perturbations. The discrete nature of such perturbations makes gradient-based optimization algorithms less efficient, and thus more difficult to learn effective adversarial cues. Therefore, how to design an optimization strategy for discrete text is a challenge.

[0059] The embodiment of the present invention first designs a perturbation strategy of adding sentence-end suffixes based on text embedding, perturbs the clean prompts in the text embedding space, and explores the loopholes of the text graph model in the hidden generation of risk targets without changing the main semantics of the clean prompts; then proposes an objective function that matches the multimodal features of the text in the feature space of the CLIP text-text multimodal model to associate adversarial prompts and risk target images; in order to solve the problem of difficulty in optimizing discrete text perturbations, the embodiment of the present invention proposes an optimization method based on proxy embedding to achieve an efficient gradient-based optimization process and obtain the final adversarial prompts; finally, based on the clean prompts and the learned adversarial prompts, the embodiment of the present invention proposes a text classification model to detect and filter adversarial prompts and enhance the protection of the text graph model. The embodiment of the present invention realizes the generation of adversarial prompts without relying on a specific text graph model, and can realize the hidden generation of risk target images in multiple text graph models. Therefore, the classification model learned by using these adversarial prompts can effectively detect adversarial prompts on multiple text graph models, thereby enhancing the performance of model security protection measures, preventing the output of harmful images, and meeting various needs in practical applications.

[0060] Example 1

[0061] A security improvement method for text graph models based on adversarial hint mining, see Figure 1 , the method mainly comprises the following steps:

[0062] 101: Taking the clean prompt describing the non-risk image as the first input, the discrete clean prompt is mapped to the continuous vector space through the embedding layer in the CLIP multimodal model to obtain the corresponding clean embedding;

[0063] The clean prompt is a text prompt describing a non-risky image. Step 101 includes:

[0064] 1) Use the tokenizer in the CLIP model to divide the input clean prompt into several tokens;

[0065] 2) Use the embedding layer in the CLIP model to map the word units corresponding to the clean prompts into multiple clean embeddings.

[0066] In order to take into account the difficulty of generating risky target images while disturbing clean prompts, an embodiment of the present invention proposes a perturbation strategy for adding a suffix to the end of a sentence based on text embedding.

[0067] 102: Predefine several optimizable noise embeddings as optimization targets, and use the nearest neighbor search method to obtain the proxy embedding closest to the noise embedding in the embedding space, and splice it to the tail of the clean embedding to obtain the adversarial embedding, and then use the text encoder in the CLIP model to encode the adversarial embedding to extract the features of the adversarial prompt;

[0068] Wherein, the step 102 includes:

[0069] 1) Predefine several optimizable noise embeddings;

[0070] 2) Use nearest neighbor search to search for the proxy embedding closest to the noise embedding in the embedding space;

[0071] 3) Concatenate the proxy embedding to the end of the clean embedding to obtain the adversarial embedding;

[0072] 4) Utilize the text encoder in the CLIP model to encode the adversarial embedding to extract the features of the adversarial prompt.

[0073] In order to establish the association relationship between the adversarial prompt and the risk target image, the embodiment of the present invention proposes to use the image-text matching target loss in the CLIP multimodal large model.

[0074] 103: Taking the risk target as the second input, using the arbitrary text image model to collect several risk target images, and using the image encoder in the CLIP model to extract risk target image features;

[0075] Wherein, the step 103 includes:

[0076] 1) Using any text-generated graph model, taking the risk target as input, a number of risk target images are generated;

[0077] 2) Use the image encoder in the CLIP model to extract features of risky target images.

[0078] 104: Using a graph-text matching objective function, the semantic similarity between the adversarial prompt feature and the risk target image feature is maximized to establish the association relationship between the adversarial prompt and the risk target image;

[0079] In order to overcome the problem of difficulty in optimizing discrete text, an embodiment of the present invention proposes an optimization method based on proxy embedding, which optimizes noise embedding using gradient information.

[0080] 105: In the process of reversely optimizing the noise embedding using the objective function, the gradient corresponding to the proxy embedding is passed to the noise embedding construction optimization process to achieve gradient-based continuous optimization of the noise embedding;

[0081] Wherein, the step 105 includes:

[0082] 1) Calculate the gradient of the proxy embedding using the image-text matching objective function in step 104;

[0083] 2) The gradient information of the proxy embedding is passed to the noise embedding, and a gradient update operation is performed to optimize the noise embedding.

[0084] 106: After the optimization is completed, a word search space is defined and the word that best matches the noise embedding is queried in the search space and concatenated to the clean prompt to generate the final adversarial prompt.

[0085] Wherein, the step 106 includes:

[0086] 1) Define the word search space and determine which words can be candidates;

[0087] 2) Use the noise embedding to search for the best matching word in the search space and concatenate it to the end of the clean prompt to obtain

[0088] and

[0089] Adversarial cue corresponding to the target image.

[0090] In order to improve the security of the text graph model against adversarial prompts, an embodiment of the present invention proposes to use the learned adversarial prompts to build a text classifier to detect adversarial prompts.

[0091] 107: Based on the clean prompts and the learned adversarial prompts, the embodiment of the present invention trains a text classification model to detect and filter adversarial prompts, enhance the security protection performance of the text-generated graph model, and ensure that the text-generated graph model cannot generate risky target images.

[0092] Wherein, the step 107 includes:

[0093] 1) Design the text classification model architecture, which mainly includes: text prompt feature extractor and feature classifier;

[0094] 2) Using clean cues and adversarial cues as data benchmarks, the classifier model is trained using binary classification loss.

[0095] In summary, the embodiments of the present invention utilize a sentence-end suffix adding strategy based on text embedding to realize the covert generation of risk target images without changing the main semantics of the clean prompt, thereby alleviating the problem of constraints between the disturbance intensity and the difficulty of generating risk target images; an optimization algorithm based on proxy embedding is used to alleviate the problem of difficulty in optimizing discrete text, and gradient information is used to efficiently optimize noise embedding; by utilizing the image-text matching objective function in the CLIP multimodal large model, an association relationship between the adversarial prompt and the target image is established, thereby realizing the generation of adversarial prompts for risk targets without relying on a specific text-based graph model; using clean prompts and adversarial prompts as data benchmarks, a classification model is designed, and learning is optimized with a binary classification loss constraint model, thereby realizing detection and filtering of adversarial prompts, enhancing the performance of security protection of the text-based graph model, and ensuring that the text-based graph model cannot generate risk target images.

[0096] Example 2

[0097] The scheme in Example 1 is further introduced below in combination with specific examples and calculation formulas, as described below for details:

[0098] 201: Take the clean prompt as input, map the discrete clean prompt to the continuous vector space through the embedding layer in the CLIP multimodal model, and obtain the corresponding clean embedding;

[0099] The embodiment of the present invention utilizes the word segmenter T in the CLIP model to firstly divide the input clean prompt x into K word units:

[0100] T(x)={x 1 ,x 2 ,...,x K} (1)

[0101] The embodiment of the present invention uses the embedding layer Emb in the CLIP model to map K word units to the embedding space to obtain a clean embedding:

[0102]

[0103] Among them, E x For clean embedding corresponding to clean prompts, is the kth word x k The corresponding embedding.

[0104] In order to take into account the difficulty of generating risky target images while disturbing clean prompts, an embodiment of the present invention proposes a perturbation strategy for adding a sentence end suffix based on text embedding.

[0105] 202: Predefine several optimizable noise embeddings as optimization targets, and use the nearest neighbor search method to obtain the proxy embedding closest to the noise embedding in the embedding space, and splice it to the tail of the clean embedding to obtain the adversarial embedding, and then use the text encoder in the CLIP model to encode the adversarial embedding to extract the features of the adversarial prompt;

[0106] The embodiment of the present invention first predefines M optimizable embeddings as noise embeddings E n :

[0107]

[0108] in, is the mth embedding in the noise embedding.

[0109] Since the CLIP model only accepts fixed-form text embeddings, the embodiment of the present invention uses a nearest neighbor search algorithm to obtain the text embedding that best matches the noise embedding in the embedding space, called the proxy embedding E p :

[0110]

[0111] Among them, E s is the text embedding of all words accepted by the CLIP model, is the mth embedding in the noise embedding, The CLIP model can accept text embeddings and Most similar embedding.

[0112] Then, the embodiment of the present invention concatenates the proxy embedding to the end of the clean embedding to construct the adversarial embedding

[0113]

[0114] Next, the embodiment of the present invention uses the text encoder in the CLIP model Encoding adversarial embedding to obtain adversarial hint features

[0115]

[0116] In order to establish the association relationship between the adversarial prompt and the risk target image, the embodiment of the present invention proposes to use the image-text matching target loss in the CLIP multimodal large model.

[0117] 203: Taking the risk target as the second input, using the arbitrary text image model to collect a number of risk target images, and using the image encoder in the CLIP model to extract risk target image features;

[0118] Among them, the risk target pt As the second input, an arbitrary text-generated graph model DM is used to generate U risk target images:

[0119]

[0120] in, is the u-th risk target image.

[0121] The embodiment of the present invention utilizes the image encoder in the CLIP model Extract risk target image features f pt :

[0122]

[0123] in, is the feature corresponding to the u-th risk target image.

[0124] 204: Using a graph-text matching objective function, the semantic similarity between the adversarial prompt feature and the risk target image feature is maximized to establish the association relationship between the adversarial prompt and the risk target image;

[0125] Among them, the embodiment of the present invention uses an image-text matching objective function to maximize the similarity between the adversarial prompt feature and the risk target image feature:

[0126]

[0127] In order to overcome the problem of difficulty in optimizing discrete text, an embodiment of the present invention proposes an optimization method based on proxy embedding, which optimizes noise embedding using gradient information.

[0128] 205: In the process of reversely optimizing the noise embedding using the objective function, the gradient corresponding to the proxy embedding is passed to the noise embedding construction optimization process to achieve gradient-based continuous optimization of the noise embedding.

[0129] Among them, the embodiment of the present invention first calculates the gradient corresponding to the proxy embedding according to the image-text matching objective function:

[0130]

[0131] Secondly, the embodiment of the present invention passes the gradient corresponding to the proxy embedding to the noise embedding and performs a gradient update operation:

[0132]

[0133] Among them, η is the learning rate hyperparameter for gradient update.

[0134] 206: After the optimization is completed, a word search space is defined, and the word that best matches the noise embedding is queried in the search space and concatenated to the clean prompt to generate the final adversarial prompt.

[0135] The embodiment of the present invention modifies the word space Vocab of the CLIP model twice to define the word search space S of the embodiment of the present invention.

[0136] The first modification is to remove all non-English words from the CLIP model word space to ensure that the adversarial prompts contain clear semantic words:

[0137] S={t v |t v ∈Vocab,t v ∈English} (12)

[0138] The second modification deleted all synonyms related to risk targets to ensure that the adversarial prompts do not contain words related to risk targets, thereby facilitating the discovery of model vulnerabilities.

[0139] Among them, the embodiment of the present invention uses a synonym model Word2Vec to obtain the risk target p t And use the word segmenter T in the CLIP model to divide these synonyms into several tokens:

[0140] W pt =T(Word2Vec(p t )) (13)

[0141] The embodiment of the present invention defines the word search space S as:

[0142]

[0143] The embodiment of the present invention uses the nearest neighbor algorithm to search for the word that best matches the optimized noise embedding in the search space as the suffix of the clean prompt:

[0144]

[0145] in, is the mth noise embedding learned after optimization, Emb(t m ) is the text embedding E that the CLIP model can accept s Zhongyu The most similar embedding, t m Embed the most similar text Embed(t m ) corresponds to the word.

[0146] Finally, the embodiment of the present invention appends the learned word unit to the clean prompt to obtain the final adversarial prompt:

[0147]

[0148] In order to improve the security of the text graph model against adversarial prompts, an embodiment of the present invention proposes to use the learned adversarial prompts to build a text classifier to detect adversarial prompts.

[0149] 207: Based on the clean prompts and the learned adversarial prompts, the embodiment of the present invention trains a text classification model to detect and filter adversarial prompts, enhance the security protection performance of the text-generated graph model, and ensure that the text-generated graph model cannot generate risky target images.

[0150] Among them, the text classification model contains two main modules, CLIP text feature encoder It is used to extract the features of the input text prompt, and the classifier fc is used to classify and identify the extracted text features.

[0151] The training process of the text classification model is as follows: As a data benchmark, the parameter update of the text classification model is constrained by the binary classification loss:

[0152]

[0153] Among them, N adv is the number of samples of adversarial prompts, N cln is the number of clean prompt samples, y n is the label corresponding to the nth sample in the dataset, It is the probability value predicted by the text classifier for the nth sample.

[0154] After training the text classification model, deploy it in the text graph model security protection module to improve the model's security protection capabilities against harmful inputs. Specifically, for any user input x, obtain the predicted probability value of its text classification model When the predicted probability value , it is judged as harmful input and the Wensheng graph model is prohibited from generating images.

[0155] Example 3

[0156] The feasibility of the solutions in Embodiments 1 and 2 is verified by combining specific examples, as described below:

[0157] The embodiment of the present invention first generates 5000 adversarial prompts using the proposed method to form an adversarial prompt dataset; then, a security assessment experiment is conducted on four currently popular text graph models to evaluate the effectiveness of the adversarial prompts; finally, the trained text classification model is deployed in the four text graph models, and the security of the text graph model after the classification model is deployed is evaluated.

[0158] 1. Construction of adversarial prompt dataset

[0159] The embodiment of the present invention first collects 100 clean prompts describing regular objects and 10 risk targets. For each clean prompt and risk target, the embodiment of the present invention uses the proposed adversarial prompt generation method to generate 5 suffixes of 1-5 words in length, which are spliced ​​at the end of the clean prompt to form 5 adversarial prompts. In summary, the embodiment of the present invention constructs a data set containing 5000 adversarial prompts.

[0160] 2. Security assessment test of Wensheng graph model

[0161] The embodiment of the present invention uses the constructed adversarial prompt dataset to evaluate the security of four popular Wensheng graph models: Stable Diffusion V1.4, Stable Diffusion V1.5, Stable Diffusion V2.1, and Stable Diffusion XL. The embodiment of the present invention uses the success rate indicator (SR) to evaluate whether the adversarial prompt is effective in generating risk images. The higher the SR, the more effective the adversarial prompt is.

[0162] Table 1 shows the success rate of the adversarial prompts generated by the embodiment of the present invention on four popular text graph models. From the results, it can be seen that the embodiment of the present invention only needs to generate one word suffix to be attached to the end of the clean prompt, that is, a certain degree of success rate can be achieved. Moreover, when the embodiment of the present invention generates more suffix words, the success rate is greatly improved, which further verifies the effectiveness of the embodiment of the present invention. Furthermore, the adversarial prompts generated by the embodiment of the present invention have an effective success rate on the four text graph models, which verifies the generalization and versatility of the embodiment of the present invention.

[0163] Table 1. Success rates on four popular text graph models

[0164]

[0165] 3. Security assessment test of the text graph model after deploying the text classification model

[0166] The embodiment of the present invention uses the trained text classification model deployed in the above four text graph models. After the user inputs the prompt, the text classification model first determines whether the input is risky. When there is a risk, it is no longer sent to the text graph model for image generation to protect the security of the model. In order to evaluate the effectiveness of the text classification model, the embodiment of the present invention sends the above 5,000 adversarial prompts to the text graph model after the text classification model is deployed for security assessment testing. The evaluation results are shown in Table 2. It can be seen that after the text classification model is deployed, the probability of the adversarial prompt effectively generating a risk image is greatly reduced compared to Table 1, which verifies that the text classification model of the present invention can effectively improve the security of the text graph model.

[0167] Table 2. Success rates on four popular text graph models after deploying the text classification model

[0168]

[0169]

[0170] Example 4

[0171] A device for improving the security of a text image based on adversarial prompt mining, the device comprising: a processor and a memory, the memory storing program instructions, the processor calling the program instructions stored in the memory to enable the device to execute the following method steps in embodiment 1:

[0172] Predefine several optimizable noise embeddings as optimization targets, and use the nearest neighbor search method to obtain the proxy embedding closest to the noise embedding in the embedding space, and splice it to the tail of the clean embedding to obtain the adversarial embedding. Use the text encoder in the CLIP model to encode the adversarial embedding to extract the features of the adversarial prompt.

[0173] By using the image-text matching objective function, the semantic similarity between the adversarial prompt features and the risk target image features is maximized, and the association relationship between the adversarial prompt and the risk target image is established;

[0174] In the process of reversely optimizing the noise embedding using the objective function, the gradient corresponding to the proxy embedding is passed to the noise embedding construction optimization process to achieve gradient-based continuous optimization of the noise embedding;

[0175] After the optimization is completed, the word that best matches the noisy embedding is queried in the word search space and concatenated to the clean prompt to generate the final adversarial prompt.

[0176] Among them, the noise is embedded in E n To: predefine M optimizable embeddings;

[0177]

[0178] in, is the mth embedding in the noise embedding;

[0179] Agent Embedding E p To do this: use nearest neighbor search to obtain the text embedding that best matches the noise embedding in the embedding space;

[0180]

[0181] Among them, E s is the text embedding of all words accepted by the CLIP model, is the mth embedding in the noise embedding, The CLIP model can accept text embeddings and Most similar embeddings;

[0182] Adversarial Embedding To do this: concatenate the proxy embedding to the end of the clean embedding;

[0183]

[0184] Among them, E x For clean embedding.

[0185] Among them, the image-text matching objective function is:

[0186]

[0187] in, is the feature corresponding to the u-th risk target image, is the adversarial cue feature, and U is the number of risky target images.

[0188] Among them, the gradient corresponding to the proxy embedding is:

[0189]

[0190] Pass the gradient corresponding to the proxy embedding to the noise embedding and perform a gradient update operation:

[0191]

[0192] Among them, η is the learning rate hyperparameter for gradient update.

[0193] Among them, the word that best matches the noise embedding is queried in the word search space and concatenated to the clean prompt to generate the final adversarial prompt:

[0194] The first modification deleted all non-English words in the CLIP model word space;

[0195] S={t v |t v ∈Vocab,tv ∈ English}

[0196] The second revision deleted all synonyms related to risk objectives;

[0197] Using a synonym model Word2Vec to obtain the risk target p t And use the word segmenter T in the CLIP model to divide these synonyms into several tokens:

[0198] W pt =T(Word2Vec(p t ))

[0199] Define the word search space S as:

[0200]

[0201] Search the search space for the word that best matches the optimized noisy embedding as the suffix of the clean prompt:

[0202]

[0203] in, is the mth noise embedding learned after optimization, Emb(t m ) is the text embedding E that the CLIP model can accept s Zhongyu The most similar embedding, t m Embed the most similar text Embed(t m ) corresponding to the word;

[0204] Append the learned tokens to the clean prompt to obtain the final adversarial prompt:

[0205]

[0206] The device also includes: training a text classification model, the text classification model includes: CLIP text feature encoder It is used to extract the features of the input text prompt, and the classifier fc is used to classify and identify the extracted text features;

[0207] With clean prompt x and learned adversarial prompt As a data benchmark, the parameter update of the text classification model is constrained by the binary classification loss:

[0208]

[0209] Among them, N adv is the number of samples of adversarial prompts, N cln is the number of clean prompt samples, yn is the label corresponding to the nth sample in the dataset, It is the probability value predicted by the text classifier for the nth sample.

[0210] It should be pointed out here that the device description in the above embodiment corresponds to the method description in the embodiment, and the embodiment of the present invention will not be described in detail here.

[0211] The execution subjects of the above-mentioned processor and memory can be devices with computing functions such as computers, single-chip microcomputers, and microcontrollers. In specific implementation, the embodiments of the present invention do not limit the execution subjects and are selected according to the needs of actual applications.

[0212] The data signal is transmitted between the memory and the processor via a bus, which is not described in detail in the embodiment of the present invention.

[0213] Based on the same inventive concept, an embodiment of the present invention further provides a computer-readable storage medium, the storage medium includes a stored program, and when the program is running, the device where the storage medium is located is controlled to execute the method steps in the above embodiment.

[0214] The computer-readable storage medium includes but is not limited to a flash memory, a hard disk, a solid-state drive, and the like.

[0215] It should be pointed out here that the description of the readable storage medium in the above embodiment corresponds to the description of the method in the embodiment, and the embodiment of the present invention will not be described in detail here.

[0216] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions according to the embodiments of the present invention are generated.

[0217] The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions may be stored in a computer-readable storage medium or transmitted via a computer-readable storage medium. The computer-readable storage medium may be any available medium that can be accessed by the computer or a data storage device such as a server or a data center that includes one or more available media. The available medium may be a magnetic medium or a semiconductor medium, etc.

[0218] Unless otherwise specified, the models of the components in the embodiments of the present invention are not limited, and any device that can perform the above functions may be used.

[0219] Those skilled in the art can understand that the accompanying drawing is only a schematic diagram of a preferred embodiment, and the serial numbers of the embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments.

[0220] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

[0221] References:

[0222] [1]Sohl-Dickstein, J., Weiss, E., Maheswaranathan, N. & Ganguli, S. Deep unsupervised learning using nonequilibrium thermodynamics. In International conference on machine learning, 2256-2265 (PMLR, 2015).

[0223] [2]Li,B.,Qi,X.,Lukasiewicz,T.&Torr,P.Controllable text-to-imagegeneration.Adv.Neural Inf.Process.Syst.32(2019)

[0224] [3]Nichol,A.et al.Glide:Towards photorealistic image generation and editing with text-guided diffusion models.Prepr.At https: / / arXiv.org / 2112.10741(2021)

[0225] [4]OpenAI.Dall·e 3(2023).https: / / openai.com / index / dall-e-3.

[0226] [5]Midjourney,I.Midjourney(2022).https: / / midjourney.com / .

[0227] [6]Rombach,R.,Blattmann,A.,Lorenz,D.,Esser,P.&Ommer,B.High-resolutionimage synthesis with latent diffusion models.In Proceedings of the IEEE / CVFconference on computer vision and pattern recognition,10684-10695(2022)

[0228] [7]Ananya.Aiimage generators often give racist and sexist results:canthey be fixed?NEWS FEATURE 627,722-725(2024).DOI 10.1038 / d41586-024-00674-9.

[0229] [8]Guo,D.,Chen,H.,Wu,R.&Wang,Y.Aigc challenges and opportunitiesrelated to public safety:a case study of chatgpt.J.Saf.Sci.Resil.4,329-339(2023).

[0230] [9]Rando,J.,Paleka,D.,Lindner,D.,Heim,L.&Tram`er,F.Red-teaming thestablediffusion safety filter(2022)

[0231]

[10] Yang,Y.,Hui,B.,Yuan,H.,Gong,N.&Cao,Y.Sneakyprompt:Jailbreakingtext-to-image generative models.In 2024IEEE Symposium on Security and Privacy(SP),123-123(IEEE Computer Society,2024)

[0232]

[11] Ba,Z.et al.Surrogateprompt:Bypassing the safety filter of text-to-image models via substitution.Prepr.At https: / / arXiv.org / 2309.14122(2023)

[0233] Unless otherwise specified, the models of the components in the embodiments of the present invention are not limited, and any device that can perform the above functions may be used.

[0234] Those skilled in the art can understand that the accompanying drawing is only a schematic diagram of a preferred embodiment, and the serial numbers of the embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments.

[0235] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for improving the security of a text graph model based on adversarial hint mining, characterized in that: The method comprises: Predefine several optimizable noise embeddings as optimization targets, and use the nearest neighbor search method to obtain the proxy embedding closest to the noise embedding in the embedding space, and splice it to the tail of the clean embedding to obtain the adversarial embedding. Use the text encoder in the CLIP model to encode the adversarial embedding to extract the features of the adversarial prompt. By using the image-text matching objective function, the semantic similarity between the adversarial prompt features and the risk target image features is maximized, and the association relationship between the adversarial prompt and the risk target image is established; In the process of reversely optimizing the noise embedding using the objective function, the gradient corresponding to the proxy embedding is passed to the noise embedding construction optimization process to achieve gradient-based continuous optimization of the noise embedding; After the optimization is completed, the word that best matches the noisy embedding is queried in the word search space and concatenated to the clean prompt to generate the final adversarial prompt.

2. According to the method for improving the security of the cultural graph model based on adversarial prompt mining, it is characterized in that: The noise is embedded in E n To: predefine M optimizable embeddings; in, is the mth embedding in the noise embedding; The agent embeds E p To do this: use nearest neighbor search to obtain the text embedding that best matches the noise embedding in the embedding space; Among them, E s is the text embedding of all words accepted by the CLIP model, is the mth embedding in the noise embedding, The CLIP model can accept text embeddings and Most similar embeddings; The adversarial embedding To do this: concatenate the proxy embedding to the end of the clean embedding; Among them, E x For clean embedding.

3. According to the method for improving the security of the cultural graph model based on adversarial prompt mining, it is characterized in that: The image-text matching objective function is: in, is the feature corresponding to the u-th risk target image, is the adversarial cue feature, and U is the number of risky target images.

4. A method for improving the security of a text graph model based on adversarial prompt mining according to claim 3, characterized in that: The gradient corresponding to the proxy embedding is: Pass the gradient corresponding to the proxy embedding to the noise embedding and perform a gradient update operation: Among them, η is the learning rate hyperparameter for gradient update.

5. A method for improving the security of a text graph model based on adversarial prompt mining according to claim 1, characterized in that: The word search space is searched for the word that best matches the noise embedding, and the final adversarial prompt is generated after being concatenated with the clean prompt: The first modification deleted all non-English words in the CLIP model word space; S={t v |t v ∈Vocab,t v ∈English} The second revision deleted all synonyms related to risk objectives; Using a synonym model Word2Vec to obtain the risk target p t And use the word segmenter T in the CLIP model to divide these synonyms into several tokens: W pt =T(Word2Vec(p t )) Define the word search space S as: Search the search space for the word that best matches the optimized noisy embedding as the suffix of the clean prompt: in, is the mth noise embedding learned after optimization, Emb(t m ) is the text embedding E that the CLIP model can accept s Zhongyu The most similar embedding, t m Embed the most similar text Embed(t m ) corresponding to the word; Append the learned tokens to the clean prompt to obtain the final adversarial prompt:

6. A method for improving the security of a text graph model based on adversarial prompt mining according to claim 1, characterized in that: The method further comprises: training a text classification model, wherein the text classification model comprises: a CLIP text feature encoder It is used to extract the features of the input text prompt, and the classifier fc is used to classify and identify the extracted text features; With clean prompt x and learned adversarial prompt As a data benchmark, the parameter update of the text classification model is constrained by the binary classification loss: Among them, N adv is the number of samples of adversarial prompts, N cln is the number of clean prompt samples, y n is the label corresponding to the nth sample in the dataset, It is the probability value predicted by the text classifier for the nth sample.

7. A device for improving the security of a text graph model based on adversarial prompt mining, characterized in that: The device comprises: a processor and a memory, wherein program instructions are stored in the memory, and the processor calls the program instructions stored in the memory to enable the device to execute any one of the methods of claims 1-6.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor is enabled to perform the method according to any one of claims 1 to 6.

Images