Large model sensitive expression fuzzification processing method and system and computer equipment

Abstract

The invention relates to a large model sensitive expression fuzzification processing method and system and computer equipment, and the method comprises the steps: obtaining an original cue word; based on the context information of the original cue word, expanding and compressing the original cue word to obtain a first cue word; performing semantic replacement processing on sensitive words in the first cue word to generate a second cue word; and performing semantic consistency enhancement and expansion processing on the second cue word to obtain a target cue word. Through the method and the device, the original cue word can be expanded and compressed in combination with the context information, the semantic replacement and semantic consistency enhancement processing of the sensitive word can be performed, and the sensitive expression in the original cue word can be fuzzified, so that the more effective target cue word can be generated; and the safety protection of the large model can be better improved by utilizing the target cue word.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method, system and computer device for fuzzy processing of sensitive expressions of large models. Background Art

[0002] At present, large language models have shown broad application prospects in many fields. In natural language processing (NLP) tasks such as text generation, machine translation, and question-answering systems, large language models have shown excellent performance. With the rapid progress of large language models, jailbreak attacks are a type of attack against large language models that attempts to bypass the model's established limitations or defense mechanisms. By constructing specific prompt words, the model is induced to generate non-compliant content, thereby destroying the model's intended function.

[0003] In order to improve the security of large language models, we can currently perform security checks on the models through simulated attacks or improve the model's security protection mechanism. However, the current model's security protection mechanism can easily detect prompt words containing sensitive expressions, so a more effective prompt word is needed. Summary of the invention

[0004] Based on this, it is necessary to provide a large-model sensitive expression fuzzy processing method, system and computer device that can generate more effective prompt words to address the above technical problems.

[0005] In the first aspect, a large model sensitive expression fuzzification processing method is provided in this embodiment, including:

[0006] Get the original prompt word;

[0007] Based on the context information of the original prompt word, the original prompt word is expanded and compressed to obtain a first prompt word;

[0008] Performing semantic replacement processing on the sensitive words in the first prompt word to generate a second prompt word;

[0009] The second prompt word is processed for semantic consistency enhancement and expansion to obtain a target prompt word.

[0010] In some embodiments, the expanding and compressing the original prompt word based on the context information of the original prompt word to obtain the first prompt word includes:

[0011] Extracting keywords from the original prompt words, and determining context information corresponding to each of the keywords;

[0012] Filtering and obtaining target context information according to the correlation between the keyword and the context information;

[0013] Generate a fuzzy prompt word based on the target context information and the original prompt word;

[0014] Based on a preset coverage rate, the fuzzy prompt word is compressed to generate the first prompt word.

[0015] In some embodiments, the step of performing semantic replacement processing on the sensitive words in the first prompt word to generate the second prompt word includes:

[0016] Performing a sensitivity evaluation on each word in the first prompt word, and determining a sensitive word in the first prompt word according to the sensitivity score;

[0017] Determine a set of semantic replacement words corresponding to the sensitive words;

[0018] According to the semantic similarity between the sensitive word and the semantic replacement word, a target replacement word is obtained by screening;

[0019] Based on the target replacement word, the sensitive word is replaced with random probability multiple times to generate the second prompt word.

[0020] In some embodiments, the performing semantic consistency enhancement and expansion processing on the second prompt word to obtain the target prompt word includes:

[0021] Performing semantic consistency enhancement processing on the second prompt word based on a genetic algorithm to obtain a third prompt word;

[0022] The third prompt word is expanded based on the fine-tuned language model to obtain the target prompt word.

[0023] In some embodiments, the performing semantic consistency enhancement processing on the second prompt word based on a genetic algorithm to obtain a third prompt word includes:

[0024] constructing an initial population of a genetic algorithm based on the second prompt word;

[0025] Based on a preset fitness function, the initial population is iteratively updated to determine the third prompt word; the preset fitness function is determined by the simulated attack success rate of the second prompt word and the semantic similarity between the first prompt word and the second prompt word.

[0026] In some embodiments, the step of expanding the third prompt word based on the fine-tuning language model to obtain the target prompt word includes:

[0027] Pre-training to obtain the fine-tuning language model;

[0028] Inputting the third prompt word into the fine-tuning language model to generate an extended prompt word;

[0029] The target prompt word is obtained by screening according to the semantic distance between the extended prompt word and the original prompt word.

[0030] In some of the embodiments, it also includes:

[0031] Based on the original prompt words and the target prompt words, a safety protection mechanism of a large model is trained.

[0032] In the second aspect, a large model sensitive expression fuzzy processing system is provided in this embodiment, including:

[0033] An acquisition module is used to acquire the original prompt words;

[0034] The prompt word processing module is used to expand and compress the original prompt word based on the context information of the original prompt word to obtain a first prompt word; perform semantic replacement processing on the sensitive words in the first prompt word to generate a second prompt word; perform semantic consistency enhancement and expansion processing on the second prompt word to obtain a target prompt word.

[0035] In a third aspect, a computer device is provided in this embodiment, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the large model sensitive expression fuzzification processing method described in the first aspect is implemented.

[0036] In a fourth aspect, a storage medium is provided in this embodiment, on which a computer program is stored, and when the program is executed by a processor, the large model sensitive expression fuzzification processing method described in the first aspect is implemented.

[0037] Compared with the related art, the large model sensitive expression fuzzification processing method, system and computer device provided in this embodiment obtains the original prompt word; based on the context information of the original prompt word, the original prompt word is expanded and compressed to obtain the first prompt word; the sensitive word in the first prompt word is semantically replaced to generate the second prompt word; the second prompt word is semantically enhanced and expanded to obtain the target prompt word. Through this embodiment, the original prompt word can be expanded and compressed in combination with the context information, and the sensitive word is semantically replaced and semantically consistent. The sensitive expression in the original prompt word is fuzzified to generate a more effective target prompt word, and the target prompt word can be used to better improve the security protection of the large model.

[0038] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0040] Figure 1 is a hardware structure block diagram of a terminal of a large model sensitive expression fuzzy processing method in an embodiment;

[0041] Figure 2 is a flow chart of a method for fuzzifying sensitive expressions of a large model in an embodiment;

[0042] Figure 3 is a flow chart of a method for fuzzifying sensitive expressions of a large model in another embodiment;

[0043] Figure 4 The structure block diagram of a large model sensitive expression fuzzy processing system in an embodiment.

[0044] In the figure: 102, processor; 104, memory; 106, transmission device; 108, input and output device; 10, acquisition module; 20, prompt word processing module. DETAILED DESCRIPTION

[0045] In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0046] Unless otherwise defined, the technical terms or scientific terms involved in this application shall have the general meaning understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the", "these" and the like in this application do not represent quantitative restrictions, and they can be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether directly or indirectly. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships. For example, "A and / or B" can mean: A exists alone, A and B exist at the same time, and B exists alone. Usually, the character " / " indicates that the objects associated with each other are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0047] The method embodiment provided in this embodiment can be executed in a terminal, a computer or a similar computing device. For example, running on a terminal, Figure 1 : is a hardware structure block diagram of a terminal of the large model sensitive expression fuzzy processing method of this embodiment. Figure 1 As shown, the terminal may include one or more ( Figure 1 Only one is shown in the figure) processor 102 and memory 104 for storing data, wherein processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA. The above terminal may also include a transmission device 106 and an input and output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is for illustration only and does not limit the structure of the above terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0048] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the large model sensitive expression fuzzification processing method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0049] The transmission device 106 is used to receive or send data via a network. The above network includes a wireless network provided by the communication provider of the terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0050] At present, large language models have shown broad application prospects in many fields. In natural language processing tasks such as text generation, machine translation, and question-answering systems, large language models have shown excellent performance. With the rapid progress of large language models, jailbreak attacks are a type of attack against large language models that attempts to bypass the model's established limitations or defense mechanisms. By constructing specific prompt words, the model is induced to generate non-compliant content, thereby destroying the model's intended function.

[0051] The current large language model jailbreak prompts can be divided into three categories: (1) disguise, which is to change the conversation background or context while maintaining the same intent, such as converting a question-answering scenario into a game environment; (2) attention diversion, which aims to change the conversation background and intent, such as by diverting the attention of the large language model to generate a piece of text instead of directly answering malicious queries; (3) privilege escalation, which is to try to circumvent the restrictions imposed by the large language model by increasing the model's permissions so that it can handle malicious queries.

[0052] In order to improve the security of large language models, we can currently perform security checks on the models through simulated attacks or improve the model's security protection mechanism. However, the current model's security protection mechanism can easily detect prompt words containing sensitive expressions, so a more effective prompt word is needed.

[0053] In this embodiment, a large model sensitive expression fuzzy processing method is provided. Figure 2 is a flow chart of the method for fuzzifying sensitive expressions of large models in this embodiment. Figure 2 As shown, the method comprises the following steps:

[0054] Step S201, obtaining the original prompt word.

[0055] Specifically, specific original prompt words are pre-constructed, which usually include some malicious and misleading content, to induce the big model to output non-compliant content, thereby detecting and improving the security of the big model.

[0056] Step S202: Based on the context information of the original prompt word, the original prompt word is expanded and compressed to obtain a first prompt word.

[0057] Specifically, an open knowledge base is used to expand the context information of the original prompt word, and the context information with higher relevance is selected to expand the original prompt word according to the relevance between the context information and the original prompt word, and the sensitive content of the original prompt word is rewritten. After the expansion, the prompt word can also be compressed to obtain the first prompt word to reduce the sensitivity expressed in the original prompt word.

[0058] Step S203: semantically replace the sensitive words in the first prompt word to generate a second prompt word.

[0059] Specifically, the synonyms and hyponyms of the sensitive words in the first prompt word are searched, and the words in the set with high relevance to the sensitive words are screened to select the words with the smallest semantic deviation to perform random replacement processing on the sensitive words to generate the second prompt word. In addition, multiple second prompt words can be generated through multiple random replacement processes to form a prompt word set.

[0060] Step S204: Perform semantic consistency enhancement and expansion processing on the second prompt word to obtain a target prompt word.

[0061] Specifically, the above prompt word set is optimized using a genetic algorithm to strengthen semantic consistency, ensure that the text maintains semantic coherence and consistency between different parts, different levels or different expressions, and then expand to obtain multiple prompt words, and finally select prompt words that are not highly correlated with the original prompt words as target prompt words. In addition, there can also be multiple target prompt words, forming a target prompt word set. The target prompt words are logical and their meanings can be understood, but the sensitive expressions in them are difficult to be screened out by the large model after fuzzification, and can then be used to guide and improve the security protection mechanism of the large model.

[0062] Different from the prior art that rewrites the prompt words of the large model in terms of characters and tokens, the above steps can expand and compress the original prompt words in combination with context information, perform random semantic replacement and semantic consistency enhancement on sensitive words, and fuzzify the sensitive expressions in the original prompt words to generate effective target prompt words that are more difficult to be screened out by the large model. The target prompt words maintain logic and do not reduce the ability to understand their malicious content. Compared with the prior art, the target prompt words in this embodiment do not need to understand the internal knowledge of the large model to generate target prompt words, and are more versatile. There is no need to create a context environment for the large model to enter a specific hypothesis to ignore sensitive prompt words, which greatly reduces the cost.

[0063] In some embodiments, the above step S202 expands and compresses the original prompt word based on the context information of the original prompt word to obtain the first prompt word, including the following steps:

[0064] Extract keywords from the original prompt word and determine the context information corresponding to each keyword; filter and obtain target context information based on the correlation between the keywords and the context information; generate fuzzy prompt words based on the target context information and the original prompt word; compress the fuzzy prompt word based on a preset coverage rate to generate a first prompt word.

[0065] Specifically, keywords are extracted from the original prompt words, and the keywords may be nouns, verbs, adjectives, etc., and a set of context information related to each keyword is retrieved from an open knowledge base, wherein the open knowledge base includes but is not limited to encyclopedias, etc.

[0066] The cosine similarity is used to calculate the relevance between each context information and the keyword, so as to filter the target context information according to the relevance. The following is the cosine similarity R(c i,j ,r i ) is expressed as:

[0067]

[0068] Among them, r i represents the vector representation of the i-th keyword, n represents the total number of keywords; c i,j The vector representation of the jth context information of the ith keyword.

[0069] Screening to obtain R(c i,j ,r i)>∈1 as the target context information of the original prompt word. Exemplarily, the value range of constant ∈1 is above 0.7. The target context information and the original prompt word are used as the input of the context generation model to rewrite the original prompt word to generate a fuzzy prompt word. The context generation model includes but is not limited to open source language models such as GPT (Generative Pre-trained Transformer). The following is an expression for generating a fuzzy prompt word:

[0070] p1=Rewrite(p,C);

[0071] Among them, p represents the original prompt word, C represents the target context information, p1 represents the fuzzified prompt word, and Rewrite(·) represents the rewriting operation of the context generation model.

[0072] Considering that the fuzzy prompt word expanded by the rewriting operation may be long, the fuzzy prompt word can be further compressed to generate a simplified first prompt word. Specifically, a graph-based unsupervised text summarization algorithm or an open source language large model can be used for compression processing. The following is an expression for generating the first prompt word:

[0073] p2 = Compress(p1,C);

[0074] Wherein, p1 represents the fuzzified prompt word, p2 represents the first prompt word, C represents the target context information, and Compress(·) represents compression processing.

[0075] In order to ensure the content of the first prompt word after compression, a preset coverage rate can be set to ensure that the content overlap between the first prompt word and the fuzzy prompt word reaches the preset coverage rate. The following is an expression of using the preset coverage rate to filter the first prompt word:

[0076]

[0077] Among them, C coverage represents the coverage rate, p1 represents the fuzzy prompt word, p2 represents the first prompt word, and the value range of the preset coverage rate∈2 is greater than 0.5.

[0078] In this embodiment, the original prompt word is rewritten and compressed based on the context information to generate the first prompt word, which can be rewritten from a semantic level to reduce sensitive expressions in the original prompt word.

[0079] In some embodiments, the semantic replacement process of the sensitive words in the first prompt word in the above step S203 to generate the second prompt word includes the following steps:

[0080] Perform sensitivity evaluation on each word in the first prompt word, and determine the sensitive words in the first prompt word according to the sensitivity score; determine the corresponding semantic replacement word set of the sensitive words; screen and obtain the target replacement word according to the semantic similarity between the sensitive words and the semantic replacement words; based on the target replacement word, perform multiple random probability replacements on the sensitive words to generate a second prompt word.

[0081] Specifically, a sensitivity evaluation is performed on each word in the first prompt word, and a sensitivity scoring function is constructed to obtain a sensitivity score, so as to determine the sensitive words in the first prompt word according to the score. The words with S(w)>∈3 are screened as sensitive words, and the sensitive word set W={w1,w2,...,w n}, illustratively, the value range of constant ∈3 is greater than 0.6.

[0082] In one embodiment, a sensitivity scoring function of the prompt word can be constructed according to the sensitivity frequency and context sensitivity of each word in the prompt word. The following is an expression of the sensitivity scoring function S(w):

[0083] S(w)=α·Freq(w)+β·ContextSens(w);

[0084] Among them, Freq(w) represents the sensitive frequency of word w, which refers to the frequency of occurrence of word w in the sensitive vocabulary or sensitive corpus; ContextSens(w) represents the context sensitivity score of word w, which can be calculated based on a pre-trained language model, such as BERT (Bidirectional Encoder Representations from Transformers) or the GPT series open source model; α and β are weight coefficients.

[0085] For each sensitive word w i , query synonyms and hyponyms in the knowledge graph as a semantic replacement word set T i :

[0086] T i =KnowledgeGraphQuery(w i );

[0087] Among them, KnowledgeGraphQuery(·) represents the knowledge graph query operation.

[0088] Use cosine similarity to calculate the semantic replacement word set T i Each replacement word t i,j With sensitive words w i The semantic similarity of , the following is the cosine similarity Sim(t i,j ,w i ) is a computational expression:

[0089]

[0090] Among them, w i represents the vector representation of the i-th sensitive word, n represents the total number of sensitive words; t i,j Represents the vector representation of the jth replacement word for the i-th sensitive word.

[0091] For each sensitive word, the semantic replacement word with the smallest semantic deviation is determined as the target replacement word according to the semantic similarity of its semantic replacement word. The sensitive words in the first prompt word are randomly replaced multiple times with a preset replacement probability. For each sensitive word, a replacement or non-replacement is randomly selected each time, thereby generating multiple second prompt words to form a second prompt word set. The set size can be preset, and the preset replacement probability can be set to 0.5.

[0092] By extracting the sensitive word set in the first prompt word in this embodiment, selecting a replacement word with the smallest semantic deviation from the semantic replacement word set corresponding to the sensitive word to randomly replace the sensitive word, the sensitive expression in the prompt word can be further fuzzified.

[0093] In some of the embodiments, the semantic consistency enhancement and expansion processing of the second prompt word in the above step S204 to obtain the target prompt word includes the following steps:

[0094] The second prompt word is processed for semantic consistency enhancement based on a genetic algorithm to obtain a third prompt word; the third prompt word is expanded based on a fine-tuned language model to obtain a target prompt word.

[0095] Specifically, the initial population of the genetic algorithm is constructed using the set of the second prompt words, and iterative updates are made based on the initial population to strengthen the semantic consistency of the second prompt words to obtain the third prompt words. Finally, the third prompt words are expanded based on the fine-tuned language model to generate the target prompt words after the sensitive expressions of the original prompt words are fuzzified.

[0096] An initial population of the genetic algorithm is constructed based on the second prompt word; based on a preset fitness function, the initial population is iteratively updated to determine a third prompt word; the preset fitness function is determined by the simulated attack success rate of the second prompt word and the semantic similarity between the first prompt word and the second prompt word.

[0097] The set of the second prompt words is used as the initial population of the genetic algorithm GP = {gp1, gp2, ..., gp n The preset fitness function of the genetic algorithm is set according to the simulated attack success rate of the second prompt word and the semantic similarity between the first prompt word and the second prompt word. The following is an expression of the preset fitness function F(x):

[0098] F(x)=λ1·A(x)+Sim(x,x1);

[0099] Among them, A(x) is the simulated attack success rate of the prompt word individual x, that is, the proportion of successful attacks on the target large model after using this prompt word individual K times; Sim(x,x1) represents the similarity between the prompt word individual x and the first prompt word individual x1; λ1 represents the weight parameter.

[0100] The first few second prompt words in the set are retained from high to low in terms of fitness, prompt words are randomly combined at the character granularity to generate new individuals, and sensitive content in the prompt words is randomly replaced, and the replacement words come from the semantic replacement word set. After repeated multiple rounds of iterations, the prompt word individual with the highest fitness is selected as the optimization result to obtain the third prompt word.

[0101] Pre-training to obtain a fine-tuning language model; inputting the third prompt word into the fine-tuning language model to generate an extended prompt word; and screening to obtain a target prompt word according to the semantic distance between the extended prompt word and the original prompt word.

[0102] Among them, a pre-trained language model (such as GPT) is selected as the basic model, and the model is fine-tuned using the pre-built prompt word dataset (including the original prompt words and the fuzzified prompt words) to generate the following fine-tuned language model M fine , the following is the fine-tuning language model M fine An expression of the loss function Loss(θ):

[0103]

[0104] Among them, θ represents the model parameters; Loss(·) represents the loss function of the basic model; p i represents the i-th prompt word sample, N represents the number of samples, y i Represents the output of the model for the i-th prompt word sample.

[0105] Using the above-mentioned fine-tuned language model and control conditions, the third prompt word is input into the fine-tuned language model to generate extended prompt words. The control condition can be a random hyperparameter to control diversity, which is used to guide the fine-tuned model to generate additional context information or constraints when generating an extended prompt word set. It can be context information, target constraints such as the tone and style of the generated prompt words, or random parameters to control the length of the prompt words. The following is the expression of the set P4′ of extended prompt words:

[0106]

[0107] Among them, p represents the original prompt word, c i represents the control condition, and m represents the total number of control conditions.

[0108] Calculate the semantic distance Dist(p,p4) between the original prompt word and each prompt word in the extended prompt word set, and set a constant ∈4, and select Dist(p,p4)>∈4, that is, the prompt words with low correlation with the original prompt words are used as target prompt words to form the target prompt word set. For example, the value range of constant ∈4 is above 0.2. The following is an expression of the calculation of the semantic distance Dist(p,p4):

[0109] Dist(p,p4)=1-Sim(p,p4);

[0110] Wherein, p represents the original prompt word, p4 represents the prompt word in the set P4′ of extended prompt words, and Sim(·) represents the similarity.

[0111] In this embodiment, the prompt word after semantic replacement is optimized by using a genetic algorithm to strengthen semantic consistency, and then expanded to obtain multiple prompt words, and finally a prompt word with low correlation with the original prompt word is selected as the target prompt word.

[0112] In some of the embodiments, the following steps are also included:

[0113] Based on the original prompt words and the target prompt words, a security protection mechanism is trained for large models.

[0114] Specifically, a pre-trained language model (such as GPT) is selected as the basic model, and the model is fine-tuned using the set of original prompt words and target prompt words, so that the model learns the sensitive content implied in the prompt words, and the security protection mechanism of the target large model is trained, and the input content (prompt words) and output content of the target large model can be reviewed to achieve security protection. In addition, the target prompt words can also be used to simulate attacks on the target large model to perform security checks on the large model.

[0115] Through the security protection mechanism of using the generated target prompt words and original prompt words to train the model in this embodiment, the model can learn the sensitive content hidden in the prompts, thereby further improving the security of the model.

[0116] The present embodiment is described and illustrated below through preferred embodiments.

[0117] Figure 3 is a flow chart of the large model sensitive expression fuzzy processing method of this embodiment, such as Figure 3 As shown, the method comprises the following steps:

[0118] Step S301: obtain original prompt words, extract keywords from the original prompt words, and determine context information corresponding to each keyword.

[0119] Step S302, according to the correlation between the keyword and the context information, the target context information is screened and obtained; based on the target context information and the original prompt word, a fuzzy prompt word is generated; based on a preset coverage rate, the fuzzy prompt word is compressed to generate a first prompt word.

[0120] Step S303, performing sensitivity evaluation on each word in the first prompt word, determining sensitive words in the first prompt word according to the sensitivity score; and determining a set of semantic replacement words corresponding to the sensitive words.

[0121] Step S304, according to the semantic similarity between the sensitive words and the semantic replacement words, a target replacement word is screened and obtained; based on the target replacement word, the sensitive words are replaced with random probability multiple times to generate a second prompt word set.

[0122] Step S305: Perform semantic consistency enhancement processing on the second prompt word based on a genetic algorithm to obtain a third prompt word.

[0123] Step S306: Expand the third prompt word based on the fine-tuning language model to obtain a target prompt word.

[0124] Step S307, training the security protection mechanism of the large model based on the original prompt words and the target prompt words.

[0125] It should be noted that the steps shown in the above process or the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0126] In this embodiment, a large model sensitive expression fuzzification processing system is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. The terms "module", "unit", "sub-unit", etc. used below can implement a combination of software and / or hardware with predetermined functions. Although the systems described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0127] Figure 4 is a structural block diagram of the large model sensitive expression fuzzy processing system of this embodiment, such as Figure 4 As shown, the system includes:

[0128] An acquisition module 10 is used to acquire the original prompt word;

[0129] The prompt word processing module 20 is used to expand and compress the original prompt word based on the context information of the original prompt word to obtain a first prompt word; perform semantic replacement processing on the sensitive words in the first prompt word to generate a second prompt word; perform semantic consistency enhancement and expansion processing on the second prompt word to obtain a target prompt word.

[0130] Different from the prior art that rewrites the prompt words of the large model in terms of characters and tokens, the above system can expand and compress the original prompt words in combination with context information, perform random semantic replacement and semantic consistency enhancement on sensitive words, and fuzzify the sensitive expressions in the original prompt words to generate effective target prompt words that are more difficult to be screened out by the large model. The target prompt words remain logical and do not reduce the ability to understand their malicious content. Compared with the prior art, the target prompt words in this embodiment do not need to understand the internal knowledge of the large model to generate, and are more versatile. There is also no need to create a context environment for the large model to enter a specific hypothesis to ignore sensitive prompt words, which greatly reduces the cost.

[0131] In some embodiments, the prompt word processing module 20 is further used for:

[0132] Extract keywords from the original prompt word and determine the context information corresponding to each keyword; filter and obtain target context information based on the correlation between the keywords and the context information; generate fuzzy prompt words based on the target context information and the original prompt word; compress the fuzzy prompt word based on a preset coverage rate to generate a first prompt word.

[0133] In some embodiments, the prompt word processing module 20 is further used for:

[0134] Perform sensitivity evaluation on each word in the first prompt word, and determine the sensitive words in the first prompt word according to the sensitivity score; determine the corresponding semantic replacement word set of the sensitive words; screen and obtain the target replacement word according to the semantic similarity between the sensitive words and the semantic replacement words; based on the target replacement word, perform multiple random probability replacements on the sensitive words to generate a second prompt word.

[0135] In some embodiments, the prompt word processing module 20 is further used for:

[0136] The second prompt word is processed for semantic consistency enhancement based on a genetic algorithm to obtain a third prompt word; the third prompt word is expanded based on a fine-tuned language model to obtain a target prompt word.

[0137] In some embodiments, the prompt word processing module 20 is further used for:

[0138] An initial population of the genetic algorithm is constructed based on the second prompt word; based on a preset fitness function, the initial population is iteratively updated to determine a third prompt word; the preset fitness function is determined by the simulated attack success rate of the second prompt word and the semantic similarity between the first prompt word and the second prompt word.

[0139] In some embodiments, the prompt word processing module 20 is further used for:

[0140] Pre-training to obtain a fine-tuning language model; inputting the third prompt word into the fine-tuning language model to generate an extended prompt word; and screening to obtain a target prompt word according to the semantic distance between the extended prompt word and the original prompt word.

[0141] In some of the embodiments, the system further comprises a training module for:

[0142] Based on the original prompt words and the target prompt words, a security protection mechanism is trained for large models.

[0143] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0144] In this embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0145] Optionally, the computer device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0146] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and will not be repeated in this embodiment.

[0147] In addition, in combination with the large model sensitive expression fuzzification processing method provided in the above embodiment, a storage medium can also be provided in this embodiment to implement. The storage medium stores a computer program; when the computer program is executed by the processor, any large model sensitive expression fuzzification processing method in the above embodiment is implemented.

[0148] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0149] It should be understood that the specific embodiments described herein are only used to explain the application, rather than to limit it. Based on the embodiments provided in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of this application.

[0150] Obviously, the drawings are only some examples or embodiments of the present application. For ordinary technicians in the field, the present application can also be applied to other similar situations based on these drawings without creative work. In addition, it is understandable that although the work done in this development process may be complicated and lengthy, for ordinary technicians in the field, certain changes in design, manufacturing or production based on the technical content disclosed in this application are only conventional technical means and should not be regarded as insufficient content disclosed in this application.

[0151] The term "embodiment" in this application refers to a specific feature, structure or characteristic described in conjunction with the embodiment that can be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily mean the same embodiment, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. It is clearly or implicitly understood by those of ordinary skill in the art that the embodiments described in this application can be combined with other embodiments without conflict.

[0152] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of patent protection. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the scope of protection of the present application. Therefore, the scope of protection of the present application shall be subject to the attached claims.

Claims

1. A method for fuzzifying sensitive expressions of large models, characterized in that: include: Get the original prompt word; Based on the context information of the original prompt word, the original prompt word is expanded and compressed to obtain a first prompt word; Performing semantic replacement processing on the sensitive words in the first prompt word to generate a second prompt word; The second prompt word is processed for semantic consistency enhancement and expansion to obtain a target prompt word.

2. The method according to claim 1, characterized in that The step of expanding and compressing the original prompt word based on the context information of the original prompt word to obtain the first prompt word includes: Extracting keywords from the original prompt words, and determining context information corresponding to each of the keywords; Filtering and obtaining target context information according to the correlation between the keyword and the context information; Generate a fuzzy prompt word based on the target context information and the original prompt word; Based on a preset coverage rate, the fuzzy prompt word is compressed to generate the first prompt word.

3. The method according to claim 1, characterized in that The performing semantic replacement processing on the sensitive words in the first prompt word to generate a second prompt word includes: Performing a sensitivity evaluation on each word in the first prompt word, and determining a sensitive word in the first prompt word according to the sensitivity score; Determine a set of semantic replacement words corresponding to the sensitive words; According to the semantic similarity between the sensitive word and the semantic replacement word, a target replacement word is obtained by screening; Based on the target replacement word, the sensitive word is replaced with random probability multiple times to generate the second prompt word.

4. The method according to claim 1, characterized in that: The performing semantic consistency enhancement and expansion processing on the second prompt word to obtain a target prompt word includes: Performing semantic consistency enhancement processing on the second prompt word based on a genetic algorithm to obtain a third prompt word; The third prompt word is expanded based on the fine-tuned language model to obtain the target prompt word.

5. The method according to claim 4, characterized in that The performing semantic consistency enhancement processing on the second prompt word based on the genetic algorithm to obtain the third prompt word includes: constructing an initial population of a genetic algorithm based on the second prompt word; Based on a preset fitness function, the initial population is iteratively updated to determine the third prompt word; the preset fitness function is determined by the simulated attack success rate of the second prompt word and the semantic similarity between the first prompt word and the second prompt word.

6. The method according to claim 4, characterized in that The step of expanding the third prompt word based on the fine-tuning language model to obtain the target prompt word includes: Pre-training to obtain the fine-tuning language model; Inputting the third prompt word into the fine-tuning language model to generate an extended prompt word; The target prompt word is obtained by screening according to the semantic distance between the extended prompt word and the original prompt word.

7. The method according to claim 1, characterized in that Also includes: Based on the original prompt words and the target prompt words, a safety protection mechanism of a large model is trained.

8. A large model sensitive expression fuzzy processing system, characterized in that: include: An acquisition module is used to acquire the original prompt words; A prompt word processing module, used for expanding and compressing the original prompt word based on the context information of the original prompt word to obtain a first prompt word; Performing semantic replacement processing on the sensitive words in the first prompt word to generate a second prompt word; The second prompt word is processed for semantic consistency enhancement and expansion to obtain a target prompt word.

9. A computer device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to execute the large model sensitive expression fuzzification processing method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the large model sensitive expression fuzzification processing method described in any one of claims 1 to 7 are implemented.

Images