Large model sensitive expression fuzzing processing method and system and computer device

Abstract

The application relates to a large model sensitive expression fuzzification processing method and system and computer equipment, and the method comprises the following steps: obtaining an original prompt word; based on context information of the original prompt word, performing expansion and compression processing on the original prompt word to obtain a first prompt word; performing semantic replacement processing on a sensitive word in the first prompt word to generate a second prompt word; and performing semantic consistency enhancement and expansion processing on the second prompt word to obtain a target prompt word. Through the application, the original prompt word can be expanded and compressed in combination with context information, and the semantic replacement and semantic consistency enhancement processing of the sensitive word can be performed, the sensitive expression in the original prompt word is fuzzified, a more effective target prompt word is generated, and the safety protection of the large model can be better improved by using the target prompt word.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method, system and computer device for processing sensitive expressions of large models. Background Technology

[0002] Currently, large language models have shown broad application prospects in multiple fields. In Natural Language Processing (NLP) tasks such as text generation, machine translation, and question answering systems, large language models have demonstrated outstanding performance. With the rapid advancement of large language models, jailbreaking attacks are a type of attack targeting these models, attempting to bypass their established limitations or defense mechanisms. These attacks involve constructing specific prompts to induce the model to generate non-compliant content, thereby disrupting the model's intended function.

[0003] To improve the security of large language models, current methods include simulating attacks to test the model's security or improving its security protection mechanisms. However, current security protection mechanisms can easily detect prompt words containing sensitive expressions, so a more effective prompt word is needed. Summary of the Invention

[0004] Therefore, it is necessary to provide a method, system, and computer device for large-scale sensitive expression fuzzing processing that can generate more effective prompt words, addressing the aforementioned technical problems.

[0005] Firstly, this embodiment provides a method for blurring sensitive representations in large models, including:

[0006] Get the original prompt words;

[0007] Based on the context information of the original prompt word, the original prompt word is expanded and compressed to obtain the first prompt word;

[0008] The sensitive words in the first prompt word are semantically replaced to generate a second prompt word;

[0009] The second prompt word is subjected to semantic consistency enhancement and expansion processing to obtain the target prompt word.

[0010] In some embodiments, the step of expanding and compressing the original prompt word based on its contextual information to obtain a first prompt word includes:

[0011] Extract keywords from the original prompts and determine the context information corresponding to each keyword;

[0012] Target context information is obtained by filtering based on the relevance between the keywords and the context information;

[0013] Based on the target context information and the original prompt words, generate fuzzy prompt words;

[0014] Based on a preset coverage rate, the fuzzy prompt words are compressed to generate the first prompt word.

[0015] In some embodiments, the step of semantically replacing sensitive words in the first prompt word to generate a second prompt word includes:

[0016] A sensitivity assessment is performed on each word in the first prompt, and the sensitive words in the first prompt are determined based on the sensitivity score.

[0017] Determine the set of semantic replacement words corresponding to the sensitive words;

[0018] Based on the semantic similarity between the sensitive words and the semantic replacement words, target replacement words are selected.

[0019] Based on the target replacement word, the sensitive word is replaced multiple times with random probability to generate the second prompt word.

[0020] In some embodiments, the semantic consistency enhancement and expansion processing of the second prompt word to obtain the target prompt word includes:

[0021] The second prompt word is subjected to semantic consistency enhancement processing based on a genetic algorithm to obtain the third prompt word;

[0022] The target prompt word is obtained by expanding the third prompt word based on the fine-tuned language model.

[0023] In some embodiments, the step of performing semantic consistency enhancement processing on the second prompt word based on a genetic algorithm to obtain the third prompt word includes:

[0024] The initial population for the genetic algorithm is constructed based on the second prompt word;

[0025] Based on a preset fitness function, the initial population is iteratively updated to determine the third prompt word; the preset fitness function is determined by the simulated attack success rate of the second prompt word and the semantic similarity between the first prompt word and the second prompt word.

[0026] In some embodiments, the expansion of the third prompt word based on a fine-tuned language model to obtain the target prompt word includes:

[0027] The fine-tuned language model is obtained through pre-training;

[0028] The third prompt word is input into the fine-tuned language model to generate an expanded prompt word;

[0029] The target prompt words are obtained by filtering based on the semantic distance between the expanded prompt words and the original prompt words.

[0030] In some of these embodiments, it also includes:

[0031] Based on the original prompt words and the target prompt words, a security protection mechanism for a large model is trained.

[0032] Secondly, this embodiment provides a large-model sensitive expression fuzzification processing system, including:

[0033] The acquisition module is used to obtain the original prompt words;

[0034] The prompt word processing module is used to expand and compress the original prompt word based on the context information of the original prompt word to obtain a first prompt word; perform semantic replacement processing on sensitive words in the first prompt word to generate a second prompt word; and perform semantic consistency enhancement and expansion processing on the second prompt word to obtain a target prompt word.

[0035] Thirdly, this embodiment provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the large model sensitive expression fuzzification processing method described in the first aspect above.

[0036] Fourthly, this embodiment provides a storage medium storing a computer program that, when executed by a processor, implements the large model sensitive expression fuzzification processing method described in the first aspect above.

[0037] Compared with related technologies, the large-scale sensitive expression blurring processing method, system, and computer device provided in this embodiment obtains the original prompt words; expands and compresses the original prompt words based on their contextual information to obtain a first prompt word; performs semantic replacement processing on sensitive words in the first prompt word to generate a second prompt word; and performs semantic consistency enhancement and expansion processing on the second prompt word to obtain a target prompt word. Through this embodiment, by combining contextual information to expand and compress the original prompt words, as well as performing semantic replacement and semantic consistency enhancement processing on sensitive words, the sensitive expressions in the original prompt words are blurred, generating more effective target prompt words. These target prompt words can be used to better improve the security protection of large-scale models.

[0038] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. Attached Figure Description

[0039] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0040] Figure 1 This is a hardware structure block diagram of the terminal for a large model sensitive expression fuzzification processing method in one embodiment;

[0041] Figure 2 This is a flowchart of a method for handling sensitive expressions of large models in one embodiment;

[0042] Figure 3 This is a flowchart of a large model sensitive expression fuzzification processing method in another embodiment;

[0043] Figure 4 This is a block diagram of a large model-sensitive expression fuzzification processing system in one embodiment.

[0044] In the diagram: 102, processor; 104, memory; 106, transmission device; 108, input / output device; 10, acquisition module; 20, prompt word processing module. Detailed Implementation

[0045] To better understand the purpose, technical solution, and advantages of this application, the application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0046] Unless otherwise defined, the technical or scientific terms used in this application shall have the general meaning as understood by one of ordinary skill in the art to which this application pertains. Words such as “a,” “an,” “an,” “the,” “the,” and “these,” used in this application, do not indicate quantitative limitation and may be singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps or modules (units) is not limited to the listed steps or modules (units) but may include steps or modules (units) not listed, or may include other steps or modules (units) inherent to such processes, methods, products, or devices. The terms “connected,” “linked,” and “coupled,” used in this application, are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. The term “multiple” used in this application refers to two or more. The "and / or" operator describes the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: A alone, A and B simultaneously, and B alone. Typically, the character " / " indicates that the objects before and after it are in an "or" relationship. The terms "first," "second," and "third," etc., used in this application are merely for distinguishing similar objects and do not represent a specific ordering of the objects.

[0047] The method embodiments provided in this example can be executed on a terminal, computer, or similar computing device. For example, it can run on a terminal. Figure 1 This is a hardware structure block diagram of the terminal for the large model-sensitive expression fuzzification processing method in this embodiment. For example... Figure 1 As shown, a terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 and a memory 104 for storing data are also included. The processor 102 may be, but is not limited to, a microprocessor (MCU) or a programmable logic device (FPGA). The terminal may also include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that… Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the terminal described above. For example, the terminal may also include components that are larger than... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown are illustrated.

[0048] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the large model sensitive expression fuzzification processing method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer programs stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0049] The transmission device 106 is used to receive or send data via a network. This network includes a wireless network provided by the terminal's communication provider. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 can be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0050] Currently, large language models have shown broad application prospects in multiple fields. They have demonstrated outstanding performance in tasks such as text generation, machine translation, and question answering systems within natural language processing. However, with the rapid advancement of large language models, jailbreaking attacks are a type of attack targeting these models, attempting to bypass their established limitations or defense mechanisms. These attacks involve constructing specific prompts to induce the model to generate non-compliant content, thereby disrupting the model's intended functionality.

[0051] The current jailbreak prompts for large language models are mainly divided into three categories: (1) disguise, which is to change the conversation background or context while maintaining the same intent, such as turning the question-and-answer scenario into a game environment; (2) attention shift, which aims to change the conversation background and intent, such as by shifting the attention of the large language model to generate a piece of text instead of directly answering malicious queries; (3) privilege escalation, which attempts to circumvent the restrictions imposed by the large language model by escalating the model's privileges to enable it to handle malicious queries.

[0052] To improve the security of large language models, current methods include simulating attacks to test the model's security or improving its security protection mechanisms. However, current security protection mechanisms can easily detect prompt words containing sensitive expressions, so a more effective prompt word is needed.

[0053] This embodiment provides a method for blurring sensitive representations in large models. Figure 2 This is a flowchart of the large model sensitive expression fuzzification processing method in this embodiment, such as... Figure 2 As shown, the method includes the following steps:

[0054] Step S201: Obtain the original prompt words.

[0055] Specifically, certain original prompts are pre-constructed, which usually include some malicious and misleading content to induce the large model to output non-compliant content, thereby detecting and improving the security of the large model.

[0056] Step S202: Based on the context information of the original prompt words, the original prompt words are expanded and compressed to obtain the first prompt word.

[0057] Specifically, an open knowledge base is used to expand the contextual information of the original prompt words. Based on the relevance between the contextual information and the original prompt words, highly relevant contextual information is selected to expand the original prompt words. The sensitive content of the original prompt words is rewritten. After expansion, the prompt words can also be compressed to obtain the first prompt word, thereby reducing the sensitivity expressed in the original prompt words.

[0058] Step S203: Perform semantic replacement processing on the sensitive words in the first prompt word to generate the second prompt word.

[0059] Specifically, the system queries the set of synonyms and hyponyms of the sensitive words in the first prompt word, filters out words with high relevance to the sensitive words in the set, and selects the words with the least semantic deviation to randomly replace the sensitive words, generating the second prompt word. Furthermore, multiple random replacement processes can be used to generate multiple second prompt words, forming a prompt word set.

[0060] Step S204: Perform semantic consistency enhancement and expansion processing on the second prompt word to obtain the target prompt word.

[0061] Specifically, a genetic algorithm is used to optimize the aforementioned set of prompt words to enhance semantic consistency, ensuring that the text maintains semantic coherence and consistency across different parts, levels, or expressions. This is then expanded to obtain multiple prompt words, and finally, prompt words with low relevance to the original prompt words are selected as target prompt words. Furthermore, there can be multiple target prompt words, forming a target prompt word set. These target prompt words are logical and their meanings are understandable, but sensitive expressions within them are blurred, making them difficult for large models to detect. This allows them to be used to guide and improve the security protection mechanisms of large models.

[0062] Unlike existing technologies that rewrite prompts for large models based on characters and tokens, the above steps can expand and compress the original prompts by combining contextual information, perform random semantic replacement and semantic consistency enhancement on sensitive words, and blur sensitive expressions in the original prompts to generate effective target prompts that are more difficult for large models to screen out. The target prompts maintain logical consistency and do not reduce the ability to understand malicious content. Compared with existing technologies, this embodiment does not require knowledge of the internal knowledge of large models to generate target prompts, making it more versatile. Furthermore, it does not require creating a contextual environment for large models to enter a specific assumption to ignore sensitive prompts, which greatly reduces costs.

[0063] In some embodiments, step S202 above, based on the context information of the original prompt word, expands and compresses the original prompt word to obtain the first prompt word, including the following steps:

[0064] Extract keywords from the original prompts and determine the context information corresponding to each keyword; filter the target context information based on the relevance between the keywords and the context information; generate fuzzy prompts based on the target context information and the original prompts; compress the fuzzy prompts based on the preset coverage rate to generate the first prompt.

[0065] Specifically, keywords are extracted from the original prompts. Keywords can be nouns, verbs, adjectives, etc., and a set of contextual information related to each keyword is retrieved from an open knowledge base, including but not limited to encyclopedias.

[0066] Cosine similarity is used to calculate the relevance between each piece of contextual information and the keyword, allowing for the selection of target contextual information based on relevance. The following is the cosine similarity R(c) between the keyword and contextual information. i,j ,r i The computational expression of ) is as follows:

[0067]

[0068] Where, r i The vector representation of the i-th keyword, where n represents the total number of keywords; c i,j The vector representation of the j-th contextual information of the i-th keyword.

[0069] Filtering yields R(c) i,j ,r iAll contextual information ∈ 1 is used as the target contextual information for the original prompt word. For example, the value of the constant ∈ 1 is greater than or equal to 0.7. The target contextual information and the original prompt word are used as input to a context generation model to rewrite the original prompt word and generate a fuzzy prompt word. The context generation model includes, but is not limited to, open-source language models such as GPT (Generative Pre-trained Transformer). The following is one expression for generating fuzzy prompt words:

[0070] p1 = Rewrite(p, C);

[0071] Where p represents the original prompt word, C represents the target context information, p1 represents the fuzzy prompt word, and Rewrite(·) represents the rewrite operation of the context generation model.

[0072] Considering that the expanded, fuzzy prompts after rewriting may be quite long, they can be further compressed to generate simplified first prompts. Specifically, graph-based unsupervised text summarization algorithms or open-source language models can be used for compression. Below is one expression for generating the first prompt:

[0073] p2 = Compress(p1, C);

[0074] Where p1 represents the obfuscated prompt word, p2 represents the first prompt word, C represents the target context information, and Compress(·) represents compression processing.

[0075] To ensure the content of the first prompt word after compression is accurate, a preset coverage rate can be set to guarantee that the overlap between the first prompt word and the blurred prompt word reaches the preset coverage rate. Below is an example of using the preset coverage rate to filter the first prompt word:

[0076]

[0077] Among them, C coverage p1 represents the fuzzy prompt word, p2 represents the first prompt word, and the preset coverage rate ∈2 is above 0.5.

[0078] By rewriting and compressing the original prompt words based on contextual information in this embodiment to generate the first prompt word, the semantics can be rewritten, reducing the sensitive expressions in the original prompt words.

[0079] In some embodiments, step S203 above involves semantic replacement of sensitive words in the first prompt word to generate a second prompt word, including the following steps:

[0080] Sensitivity assessment is performed on each word in the first prompt word, and sensitive words in the first prompt word are determined based on the sensitivity score; the set of semantic replacement words corresponding to the sensitive words is determined; target replacement words are selected based on the semantic similarity between the sensitive words and the semantic replacement words; based on the target replacement words, the sensitive words are randomly replaced multiple times to generate the second prompt word.

[0081] Specifically, a sensitivity assessment is performed on each word in the first prompt, and a sensitivity scoring function is constructed to obtain a sensitivity score. Sensitive words in the first prompt are then determined based on these scores. Words with S(w)>∈3 are selected as sensitive words, resulting in a sensitive word set W={w1,w2,...,w...}. n For example, the value range of constant ∈3 is 0.6 and above.

[0082] In one embodiment, a sensitivity scoring function for the prompt words can be constructed based on the sensitivity frequency and contextual sensitivity of each word in the prompt words. The following is an expression of the sensitivity scoring function S(w):

[0083] S(w)=α·Freq(w)+β·ContextSens(w);

[0084] Where Freq(w) represents the sensitivity frequency of word w, which refers to the frequency of occurrence of word w in the sensitive word list or sensitive corpus; ContextSens(w) represents the context sensitivity score of word w, which can be calculated based on a pre-trained language model, such as BERT (Bidirectional Encoder Representations from Transformers) or the GPT series of open source models; α and β are weight coefficients.

[0085] For each sensitive word w i Search for synonyms and hyponyms / hypernyms in the knowledge graph as a semantic replacement word set T. i :

[0086] T i =KnowledgeGraphQuery(w i );

[0087] Among them, KnowledgeGraphQuery(·) represents the knowledge graph query operation.

[0088] Using cosine similarity to calculate the semantic replacement word set T i Each replacement word t i,j With sensitive word w i The semantic similarity is as follows: The cosine similarity Sim(t) is... i,j ,w i An expression of calculation:

[0089]

[0090] Among them, w i The vector representation of the i-th sensitive word, where n represents the total number of sensitive words; t i,j The vector representation of the j-th replacement word for the i-th sensitive word.

[0091] For each sensitive word, based on the semantic similarity of its semantic replacement word, the semantic replacement word with the smallest semantic deviation is determined as the target replacement word. Each sensitive word in the first prompt word is randomly replaced multiple times with a preset replacement probability. For each sensitive word, replacement or no replacement is randomly selected each time, thereby generating multiple second prompt words, forming a second prompt word set. The size of the set can be preset, and the preset replacement probability can be set to 0.5.

[0092] By extracting the set of sensitive words from the first prompt word in this embodiment, and selecting the replacement word with the smallest semantic deviation from the set of semantic replacement words corresponding to the sensitive word to randomly replace the sensitive word, the sensitive expression in the prompt word can be further blurred.

[0093] In some embodiments, step S204 above involves semantic consistency enhancement and expansion processing of the second prompt word to obtain the target prompt word, including the following steps:

[0094] The second prompt word is semantically consistent with the first prompt word using a genetic algorithm to obtain the third prompt word; the third prompt word is then expanded using a fine-tuned language model to obtain the target prompt word.

[0095] Specifically, the initial population of the genetic algorithm is constructed using the set of second prompt words. This initial population is then iteratively updated to enhance the semantic consistency of the second prompt words, resulting in the third prompt word. Finally, the third prompt word is expanded based on a fine-tuned language model to generate the target prompt word after sensitive expression fuzzification of the original prompt words.

[0096] An initial population for a genetic algorithm is constructed based on the second prompt word; the initial population is iteratively updated based on a preset fitness function to determine the third prompt word; the preset fitness function is determined by the simulated attack success rate of the second prompt word and the semantic similarity between the first and second prompt words.

[0097] The set of the second prompt words is used as the initial population GP = {gp1, gp2, ..., gp} for the genetic algorithm. n Based on the simulated attack success rate of the second prompt word and the semantic similarity between the first and second prompt words, a preset fitness function for the genetic algorithm is set. The following is one expression of the preset fitness function F(x):

[0098] F(x) = λ1·A(x) + Sim(x,x1);

[0099] Where A(x) is the simulated attack success rate of the prompt word individual x, that is, the proportion of successful attacks after using this prompt word individual to attack the target large model K times; Sim(x,x1) represents the similarity between the prompt word individual x and the first prompt word individual x1; λ1 represents the weight parameter.

[0100] The top few second-tier prompt words in the set are retained according to their fitness from high to low. New individuals are generated by randomly combining these prompt words at the character granularity level, and sensitive content within the prompt words is randomly replaced using words from a semantic replacement word set. After multiple iterations, the prompt word with the highest fitness is selected as the optimization result, yielding the third-tier prompt word.

[0101] A fine-tuned language model is pre-trained; the third prompt word is input into the fine-tuned language model to generate an expanded prompt word; the target prompt word is obtained by filtering based on the semantic distance between the expanded prompt word and the original prompt word.

[0102] In this process, a pre-trained language model (such as GPT) is selected as the base model, and the model is fine-tuned using a pre-built prompt word dataset (including the original prompt words and their blurred versions) to generate the following fine-tuned language model M. fine The following is a fine-tuning of the language model M. fine An expression for the loss function Loss(θ):

[0103]

[0104] Where θ represents the model parameters; Loss(·) represents the loss function of the base model; p i Let y represent the i-th prompt word sample, N represent the number of samples, and y represent the number of prompt words. i This represents the model's output for the i-th prompt word sample.

[0105] Using the aforementioned fine-tuned language model and control conditions, the third cue word is input into the fine-tuned language model to generate extended cue words. The control conditions can be random hyperparameters controlling diversity, used to guide the fine-tuned model in generating the extended cue word set with additional contextual information or constraints. These can be contextual information, target constraints such as the tone and style of the generated cue words, or random parameters controlling the length of the cue words. The following is the expression for the extended cue word set P4′:

[0106]

[0107] Where p represents the original prompt word, c i represents the control condition, and m represents the total number of control conditions.

[0108] Calculate the semantic distance Dist(p,p4) for each prompt word in the original and expanded prompt word sets, and set a constant ∈ 4. Select prompt words with Dist(p,p4) > 4 (i.e., those with low relevance to the original prompt words) as target prompt words, forming the target prompt word set. For example, the constant ∈ 4 ranges from 0.2 to above. The following is an expression for calculating the semantic distance Dist(p,p4):

[0109] Dist(p,p4) = 1 - Sim(p,p4);

[0110] Where p represents the original prompt word, p4 represents the prompt word in the set of extended prompt words P4′, and Sim(·) represents the similarity.

[0111] In this embodiment, a genetic algorithm is used to optimize the semantically replaced prompt words to enhance semantic consistency. Then, multiple prompt words are obtained by expansion, and finally, prompt words that are not highly related to the original prompt words are selected as target prompt words.

[0112] In some embodiments, the following steps are also included:

[0113] A security protection mechanism for a large model is trained based on the original prompt words and the target prompt words.

[0114] Specifically, a pre-trained language model (such as GPT) is selected as the base model. The model is then fine-tuned using a set of original and target prompt words, enabling it to learn the sensitive content implicit in the prompt words. This training yields a security protection mechanism for the target large model, allowing for the review of both the input (prompt words) and output content to achieve security protection. Alternatively, the target prompt words can be used to simulate attacks on the target large model for security checks.

[0115] The security protection mechanism in this embodiment, which utilizes the generated target prompt words and the original prompt words to train the model, enables the model to learn the sensitive content hidden in the prompts, thereby further improving the model's security.

[0116] The present embodiment will now be described and illustrated through preferred embodiments.

[0117] Figure 3 This is a flowchart of the large-model sensitive expression fuzzification processing method in this embodiment, such as... Figure 3 As shown, the method includes the following steps:

[0118] Step S301: Obtain the original prompt words, extract the keywords from the original prompt words, and determine the context information corresponding to each keyword.

[0119] Step S302: Based on the relevance of keywords and context information, target context information is obtained by filtering; based on the target context information and the original prompt words, fuzzy prompt words are generated; based on the preset coverage rate, the fuzzy prompt words are compressed to generate the first prompt word.

[0120] Step S303: Perform sensitivity assessment on each word in the first prompt word, determine the sensitive words in the first prompt word based on the sensitivity score, and determine the set of semantic replacement words corresponding to the sensitive words.

[0121] Step S304: Based on the semantic similarity between sensitive words and semantic replacement words, target replacement words are selected; based on the target replacement words, sensitive words are randomly replaced multiple times to generate a second set of prompt words.

[0122] Step S305: Perform semantic consistency enhancement processing on the second prompt word based on the genetic algorithm to obtain the third prompt word.

[0123] Step S306: Expand the third prompt word based on the fine-tuned language model to obtain the target prompt word.

[0124] Step S307: Based on the original prompt words and the target prompt words, train the security protection mechanism of the large model.

[0125] It should be noted that the steps shown in the above process or in the flowchart of the accompanying figures can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0126] This embodiment also provides a large-model sensitive expression fuzzification processing system, which is used to implement the above embodiments and preferred embodiments, and will not be repeated as described before. The terms "module," "unit," "subunit," etc., used below can refer to a combination of software and / or hardware that performs a predetermined function. Although the system described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0127] Figure 4 This is a structural block diagram of the large model-sensitive expression fuzzification processing system in this embodiment, as shown below. Figure 4 As shown, the system includes:

[0128] Module 10 is used to obtain the original prompt words;

[0129] The prompt word processing module 20 is used to expand and compress the original prompt word based on the context information of the original prompt word to obtain the first prompt word; perform semantic replacement processing on the sensitive words in the first prompt word to generate the second prompt word; and perform semantic consistency enhancement and expansion processing on the second prompt word to obtain the target prompt word.

[0130] Unlike existing technologies that rewrite prompts for large models based on characters and tokens, the above system can expand and compress the original prompts by combining contextual information, perform random semantic replacement and semantic consistency enhancement on sensitive words, and blur sensitive expressions in the original prompts to generate effective target prompts that are more difficult for large models to screen out. The target prompts maintain logical consistency and do not reduce the ability to understand malicious content. Compared with existing technologies, this embodiment does not require knowledge of the internal knowledge of large models to generate target prompts, making it more versatile. Furthermore, it does not require creating a contextual environment for large models to enter a specific assumption to ignore sensitive prompts, which greatly reduces costs.

[0131] In some embodiments, the prompt word processing module 20 described above is further configured to:

[0132] Extract keywords from the original prompts and determine the context information corresponding to each keyword; filter the target context information based on the relevance between the keywords and the context information; generate fuzzy prompts based on the target context information and the original prompts; compress the fuzzy prompts based on the preset coverage rate to generate the first prompt.

[0133] In some embodiments, the prompt word processing module 20 described above is further configured to:

[0134] Sensitivity assessment is performed on each word in the first prompt word, and sensitive words in the first prompt word are determined based on the sensitivity score; the set of semantic replacement words corresponding to the sensitive words is determined; target replacement words are selected based on the semantic similarity between the sensitive words and the semantic replacement words; based on the target replacement words, the sensitive words are randomly replaced multiple times to generate the second prompt word.

[0135] In some embodiments, the prompt word processing module 20 described above is further configured to:

[0136] The second prompt word is semantically consistent with the first prompt word using a genetic algorithm to obtain the third prompt word; the third prompt word is then expanded using a fine-tuned language model to obtain the target prompt word.

[0137] In some embodiments, the prompt word processing module 20 described above is further configured to:

[0138] An initial population for a genetic algorithm is constructed based on the second prompt word; the initial population is iteratively updated based on a preset fitness function to determine the third prompt word; the preset fitness function is determined by the simulated attack success rate of the second prompt word and the semantic similarity between the first and second prompt words.

[0139] In some embodiments, the prompt word processing module 20 described above is further configured to:

[0140] A fine-tuned language model is pre-trained; the third prompt word is input into the fine-tuned language model to generate an expanded prompt word; the target prompt word is obtained by filtering based on the semantic distance between the expanded prompt word and the original prompt word.

[0141] In some embodiments, the system further includes a training module for:

[0142] A security protection mechanism for a large model is trained based on the original prompt words and the target prompt words.

[0143] It should be noted that the above modules can be functional modules or program modules, and can be implemented through software or hardware. For modules implemented through hardware, the above modules can reside in the same processor; or the above modules can be located in different processors in any combination.

[0144] This embodiment also provides a computer device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0145] Optionally, the computer device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0146] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated in this embodiment.

[0147] Furthermore, in conjunction with the large model sensitive expression blurring processing method provided in the above embodiments, this embodiment can also provide a storage medium for implementation. This storage medium stores a computer program; when executed by a processor, the computer program implements any of the large model sensitive expression blurring processing methods in the above embodiments.

[0148] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0149] It should be understood that the specific embodiments described herein are merely illustrative of the application and not intended to limit it. All other embodiments derived by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0150] Obviously, the accompanying drawings are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar situations based on these drawings without any creative effort. Furthermore, it is understood that although the work done in this development process may be complex and lengthy, for those skilled in the art, certain design, manufacturing, or production modifications made based on the technical content disclosed in this application are merely conventional technical means and should not be considered as insufficient disclosure of this application.

[0151] The term "embodiment" in this application refers to a specific feature, structure, or characteristic described in connection with an embodiment that may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily imply the same embodiment, nor does it imply that it is mutually exclusive with or alternative to other embodiments. It will be clearly or implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0152] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of patent protection. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the appended claims.

Claims

1. A method for sensitive expression fuzzification processing of large models, characterized in that, include: Get the original prompt words; Based on the context information of the original prompt word, the original prompt word is expanded and compressed to obtain the first prompt word; The sensitive words in the first prompt word are semantically replaced to generate a second prompt word; The second prompt word is subjected to semantic consistency enhancement and expansion processing to obtain the target prompt word; Specifically, it includes the following steps: The second prompt word is semantically consistent with the second prompt word to obtain the third prompt word. This includes: constructing an initial population of the genetic algorithm based on the second prompt word; iteratively updating the initial population based on a preset fitness function to determine the third prompt word; the preset fitness function is determined by the simulated attack success rate of the second prompt word and the semantic similarity between the first prompt word and the second prompt word. The target prompt word is obtained by expanding the third prompt word based on the fine-tuned language model.

2. The method according to claim 1, characterized in that, The first prompt word is obtained by expanding and compressing the original prompt word based on the context information of the original prompt word, including: Extract keywords from the original prompts and determine the context information corresponding to each keyword; Target context information is obtained by filtering based on the relevance between the keywords and the context information; Based on the target context information and the original prompt words, generate fuzzy prompt words; Based on a preset coverage rate, the fuzzy prompt words are compressed to generate the first prompt word.

3. The method according to claim 1, characterized in that, The step of semantically replacing sensitive words in the first prompt word to generate a second prompt word includes: A sensitivity assessment is performed on each word in the first prompt, and the sensitive words in the first prompt are determined based on the sensitivity score. Determine the set of semantic replacement words corresponding to the sensitive words; Based on the semantic similarity between the sensitive words and the semantic replacement words, target replacement words are selected. Based on the target replacement word, the sensitive word is replaced multiple times with random probability to generate the second prompt word.

4. The method according to claim 1, characterized in that, The expansion of the third prompt word based on the fine-tuned language model to obtain the target prompt word includes: The fine-tuned language model is obtained through pre-training; The third prompt word is input into the fine-tuned language model to generate an expanded prompt word; The target prompt words are obtained by filtering based on the semantic distance between the expanded prompt words and the original prompt words.

5. The method according to claim 1, characterized in that, Also includes: Based on the original prompt words and the target prompt words, a security protection mechanism for a large model is trained.

6. A system for sensitive expression fuzzification processing of large models, characterized in that, include: The acquisition module is used to obtain the original prompt words; The prompt word processing module is used to expand and compress the original prompt word based on the context information of the original prompt word to obtain a first prompt word; The sensitive words in the first prompt word are semantically replaced to generate a second prompt word; The second prompt word is subjected to semantic consistency enhancement and expansion processing to obtain the target prompt word; Specifically, it includes the following steps: The second prompt word is semantically consistent with the second prompt word to obtain the third prompt word. This includes: constructing an initial population of the genetic algorithm based on the second prompt word; iteratively updating the initial population based on a preset fitness function to determine the third prompt word; the preset fitness function is determined by the simulated attack success rate of the second prompt word and the semantic similarity between the first prompt word and the second prompt word. The target prompt word is obtained by expanding the third prompt word based on the fine-tuned language model.

7. A computer device, comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to perform the large model sensitive expression fuzzification processing method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the large model sensitive expression fuzzification processing method as described in any one of claims 1 to 5.

Images