Intelligent question and answer protection method and device based on large model

Abstract

The invention relates to the technical field of large model application, and provides an intelligent question and answer protection method and device based on a large model.The method comprises the steps that a text input by a user and a safety prompt word are fused to obtain a fused prompt word, and the safety prompt word is a description text for describing a safety rule defined in a safety rule base; the fused cue words are input into a question and answer model, text content output by the question and answer model is obtained, the question and answer model is obtained through adversarial training on the basis of a training set with adversarial samples, and the adversarial samples are sample texts containing sample harmful information and sample safety cue words. The question and answer model is effectively prevented from being induced by the user to output the content containing the harmful information, and effective protection of the output content of the large model question and answer is achieved.

Description

Technical Field

[0001] The present invention relates to the field of large model application technology, and in particular to a large model-based intelligent question-answering protection method and device. Background Art

[0002] With the rapid development of artificial intelligence (AI), large language models have been widely used in natural language processing, and intelligent question-answering applications based on large models are also becoming increasingly popular. However, when large models answer questions based on user-entered prompts (i.e., reasoning), there is a risk of malicious exploitation. For example, users could use carefully crafted prompts to trick the large model into outputting harmful content (e.g., illegal content or leaking sensitive information). Therefore, it is necessary to protect the reasoning behavior of large models to prevent them from outputting harmful content under the influence of user prompts.

[0003] Most current protection methods operate at the input and output stages of large models. The input stage identifies and filters prompt words, while the output stage audits and blocks harmful content. However, the input stage can only simply identify malicious input and cannot handle logically complex input content. The output stage cannot block harmful content in real time and exhibits a lag. In short, existing large-model question-and-answer protection methods cannot effectively protect the output content (i.e., answers) of large-model question-and-answer systems. Summary of the Invention

[0004] The present invention provides a large-model-based intelligent question-answering protection method and device, which is used to solve the problem that the large-model question-answering protection method in the existing technology cannot effectively protect the output content of the large-model question-answering.

[0005] The present invention provides a large-model-based intelligent question-answering protection method, comprising the following steps: Fusing the user input text and the security prompt word to obtain a fused prompt word, wherein the security prompt word is a description text describing the security rules defined in the security rule library; The fused prompt word is input into a question-answering model to obtain text content output by the question-answering model, wherein the question-answering model is obtained through adversarial training based on a training set with adversarial samples, and the adversarial samples are sample texts containing sample harmful information and sample safety prompt words.

[0006] According to the present invention, a large-model-based intelligent question-answering protection method is provided, which fuses user input text and security prompt words to obtain fused prompt words, including: Determining a target security risk level of the user input text based on entity keywords in the user input text; Based on the target security risk level, selecting a security prompt word that matches the target security risk level from a security prompt word library; fusing the security prompt word that matches the target security risk level with the user input text to obtain the fused prompt word; The security prompt word library stores a plurality of security risk prompt words corresponding to each of the security rules, and the plurality of security risk prompt words correspond to corresponding security risk levels, respectively.

[0007] According to the present invention, a large-model-based intelligent question-answering protection method is provided, which determines the target security risk level of the user input text based on entity keywords in the user input text, including: Identify multiple entity keywords in user input text; Searching for each target risk word that matches each entity keyword in a mapping table of security risk levels and risk words; The highest security risk level among the security risk levels corresponding to the target risk words is determined as the target security risk level.

[0008] According to a large model-based intelligent question-answering protection method provided by the present invention, before the fusion prompt word is input into the question-answering model, it also includes: filtering out sensitive information in the user input text.

[0009] According to the large model-based intelligent question-answering protection method provided by the present invention, before the fusion prompt word is input into the question-answering model, the method further includes: Checking the user input text for first harmful information; In the case where the first harmful information is detected, determining replacement information corresponding to the first harmful information based on a replacement mapping table, and replacing the first harmful information with the replacement information to form a replacement text; The user input text in the fusion prompt word is replaced with the replacement text.

[0010] According to the large model-based intelligent question-answering protection method provided by the present invention, before fusing the user input text and the security prompt word to obtain the fused prompt word, the method further includes: Identify multiple entity keywords in user input text; When all entity keywords do not match the domain keywords in the whitelist, the user input text is output as not meeting the domain scope of the question-answering model, and a prompt is given to replace the user input text, wherein the whitelist specifies multiple domain keywords of the application domain of the problem handled by the question-answering model.

[0011] According to the present invention, a large-model-based intelligent question-answering protection method further comprises, after obtaining the text content output by the question-answering model: Based on the security check template library corresponding to the question-answering model, a security check is performed on the text content output by the question-answering model; the security check template library defines the second harmful information in the application field processed by the question-answering model and the corresponding avoidance strategy; When it is detected that the text content output by the question-answering model contains second harmful information, the second harmful information is processed based on the avoidance strategy.

[0012] According to a large-model-based intelligent question-answering protection method provided by the present invention, when it is detected that the text content output by the question-answering model contains second harmful information, it also includes: using the user input text to fine-tune the question-answering model.

[0013] The present invention also provides an intelligent question-answering protection device based on a large model, comprising the following modules: A prompt word fusion module is used to fuse the user input text and the security prompt word to obtain a fused prompt word, wherein the security prompt word is a description text describing the security rules defined in the security rule library; The question-answering model execution module is used to input the fusion prompt word into the question-answering model to obtain the text content output by the question-answering model, wherein the question-answering model is obtained through adversarial training based on a training set with adversarial samples, and the adversarial samples are sample texts containing sample harmful information and sample safety prompt words.

[0014] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein when the processor executes the program, it implements the large model-based intelligent question-answering protection method as described above.

[0015] The large-model-based intelligent question-answering protection method and device provided by the present invention fuse security prompt words with user input text to form fused prompt words, that is, the fused prompt words not only contain the user's questions, but also contain security prompt words used to constrain the output of the question-answering model based on the large model, so that the question-answering model can answer the questions in the user input text within the constraints of the security rules. Moreover, the question-answering model itself is obtained through adversarial training based on a training set with adversarial samples, so that it can effectively deal with logically complex user input text, and effectively prevent the question-answering model from being induced by the user to output content containing harmful information, thereby achieving effective protection of the output content of the large-model question-answering. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0017] Figure 1 It is a flow chart of the intelligent question-answering protection method based on a large model provided by the present invention.

[0018] Figure 2 It is a structural diagram of the intelligent question-answering protection device based on a large model provided by the present invention.

[0019] Figure 3 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0020] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0021] The large-model-based intelligent question-answering protection method of an embodiment of the present invention includes the following steps S110 and S120.

[0022] Step S110: The user input text and the security prompt are merged to produce a fused prompt. The security prompt is a descriptive text describing the security rules defined in the security rule library. Security rules include legal provisions and / or regulations, and the security prompt is a descriptive text explaining these legal provisions and / or regulations. Fusion combines the user input text and the security prompt into a single text description, the fused prompt.

[0023] Step S120: Input the fused prompt word into the question-answering model to obtain the text content output by the question-answering model, wherein the question-answering model is obtained through adversarial training based on a training set with adversarial samples, and the adversarial samples are sample texts containing sample harmful information and sample safety prompt words. Adversarial samples with complex logic can be used for training, so that the question-answering model can effectively deal with user input text with complex logic.

[0024] In this embodiment, adversarial samples are used to enable the question-answering model to learn and identify the characteristics of harmful information in user input text during training, and to resist the influence of harmful information on the model output, thereby avoiding outputting content containing harmful information based on user input text, thereby improving the protection capability of the question-answering model itself.

[0025] The large-model-based intelligent question-answering protection method of this embodiment fuses security prompt words with user input text to form fused prompt words. That is, the fused prompt words not only contain the user's questions, but also contain security prompt words used to constrain the output of the question-answering model based on the large model, so that the question-answering model answers the questions in the user input text within the constraints of the security rules. Moreover, the question-answering model itself is obtained through adversarial training based on a training set with adversarial samples, so that it can effectively deal with user input text with complex logic, and effectively prevent the question-answering model from being induced by the user to output content containing harmful information, thereby achieving effective protection of the output content of the large-model question-answering.

[0026] In some embodiments, step S110 specifically includes: Based on entity keywords in the user input text, a target security risk level of the user input text is determined.

[0027] Specifically, a pre-trained semantic model for analyzing security risk levels can be used to analyze the security risk level of user input text based on entity keywords in the user input text. The semantic model for analyzing security risk levels is trained using sample user input text containing entity keywords and the security risk level labels corresponding to the sample user input text.

[0028] The security risk level of each entity keyword in the user input text may also be determined through a predefined mapping relationship between entity keywords and security risk levels, thereby determining a target security risk level of the user input text.

[0029] Security risk levels are categorized as high, low, and no risk, or high, medium, low, and no risk, depending on the specific situation. For example, entity keywords such as "tax evasion," "pornography," or "fraud" are considered high risk.

[0030] Based on the target security risk level, a security prompt word that matches the target security risk level is selected from a security prompt word library, wherein the security prompt word library stores a plurality of security risk prompt words corresponding to each of the security rules, and the plurality of security risk prompt words respectively correspond to the corresponding security risk levels. That is, each security rule has different security risk prompt words corresponding to different security risk levels, and each security level corresponds to at least one security risk prompt word. For example: for user input text with a high security risk level, the corresponding security risk prompt word includes "discussion of user-raised questions is prohibited" in addition to the description of the security rule; for user input text with a low security risk level, the corresponding security risk prompt word includes "discussion of user-raised questions within the scope of relevant security rules" in addition to the description of the security rule; for user input text with no security risk, the corresponding security risk prompt word may only include a description of the security rule. Since the user input text itself has no security risk, when the question-answering model infers the user input text, the security risk prompt word also only limits the question-answering model to infer within the scope of the legal bottom line.

[0031] The security prompt word that matches the target security risk level is fused with the user input text to obtain the fused prompt word.

[0032] In this embodiment, by selecting a security prompt word that dynamically matches the security risk level of the text input by the user, the question-answering model can output text content within a reasonable range of security rules.

[0033] In some embodiments, the step of determining a target security risk level of the user input text based on entity keywords in the user input text specifically includes: Identify multiple entity keywords in the user input text. Specifically, named entity recognition (NER) technology can be used to identify multiple entity keywords in the user input text.

[0034] The mapping table between security risk levels and risk words searches for target risk words that match each entity keyword. Specifically, the cosine similarity between the risk word and the entity keyword can be calculated. If the similarity is greater than 75%, the search is considered successful. The mapping table between security risk levels and risk words records multiple risk words and their corresponding security risk levels. If the target risk word is not found for all entity keywords, the user input text is determined to contain no security risk.

[0035] The highest security risk level among the security risk levels corresponding to each target risk word is determined as the target security risk level. The security risk level of the target risk word is also the security risk level of the entity keyword that matches the target risk word. Because different entity keywords may correspond to different security risk levels, in this embodiment, the highest security risk level is selected as the target security risk level for the user input text. This can better limit the question-answering model's output of text content within reasonable security rules and avoid outputting content containing harmful information.

[0036] In some embodiments, before inputting the fused prompt word into the question-answering model, the process further includes filtering out sensitive information from the user input text. Sensitive information includes private information, such as a specific person's name, ID number, and mobile phone number, which could lead to privacy breaches. Specifically, sensitive information can be identified and removed through pattern recognition (specific patterns of ID numbers and mobile phone numbers) and named entity recognition.

[0037] Of course, this embodiment also includes filtering out some noise character strings in the user input text that interfere with model recognition through pattern recognition, so as to avoid interference with the question-answering model during reasoning.

[0038] In some embodiments, before inputting the fused prompt word into the question-answering model, the method further includes: The user input text is checked for first harmful information.

[0039] In the case where the first harmful information is detected, replacement information corresponding to the first harmful information is determined based on a replacement mapping table, and the first harmful information is replaced with the replacement information to form a replacement text.

[0040] The user input text in the fusion prompt word is replaced with the replacement text.

[0041] It should be noted that the question-answering model of this embodiment is a question-answering model based on a large model and applied to one or more specific application fields, that is, the question-answering model is only used to answer questions in the application field. Therefore, for a specific application field, a mapping table of the first harmful information in the application field and its corresponding replacement information, that is, a replacement mapping table, can be set in advance. For example: in the financial application field, the replacement information corresponding to the first harmful information "tax evasion" can be "reasonable tax avoidance", then the "tax evasion" in the user input text is replaced with "reasonable tax avoidance" to form a replacement text, and then the replacement text is used to replace the user input text in the fusion prompt word.

[0042] Of course, a semantic model based on natural language processing technology can also be used to convert the user input text containing the first harmful information into a replacement text with a similar meaning to the user input text.

[0043] In this embodiment, after integrating the safety prompt words, the first harmful information in the user input text is further replaced to further prevent the question-answering model from outputting content containing harmful information.

[0044] In some embodiments, before fusing the user input text with the security prompt word to obtain the fused prompt word, the method further includes: Identify multiple entity keywords in the user input text. Specifically, multiple entity keywords can be identified using a named entity recognition method.

[0045] When all entity keywords do not match the domain keywords in the whitelist, the user input text is output as not meeting the domain scope of the question-answering model, and a prompt is given to replace the user input text, wherein the whitelist specifies multiple domain keywords of the application domain of the problem handled by the question-answering model.

[0046] In this embodiment, before fusing the user input text and the security prompt word, the system determines whether the user input text conforms to the application domain of the questions handled by the question-answering model, and refuses to answer questions that are beyond the scope of the application domain, thereby reducing the chance of the question-answering model being maliciously exploited.

[0047] In some embodiments, after obtaining the text content output by the question-answering model, the method further includes: Based on the security check template library corresponding to the question-answer model, a security check is performed on the text content output by the question-answer model; the security check template library defines the second harmful information in the application field processed by the question-answer model and the corresponding avoidance strategy. The avoidance strategy includes replacing the output or blocking the output. Specifically, the security check template library is similar to the replacement mapping table in the above embodiment. If the second harmful information in the security check template library has corresponding replacement information, the avoidance strategy is to replace the second harmful information with the replacement information and then output it; if the security check template library does not have replacement information for the second harmful information, the avoidance strategy is to block the output.

[0048] When it is detected that the text content output by the question-answering model contains second harmful information, the second harmful information is processed based on the avoidance strategy.

[0049] Even if the user input text is processed and integrated with safety prompt words, there is still a very small probability that the question-answering model will output certain harmful information, namely, the second harmful information. In this embodiment, in order to avoid these low-probability events, the text content output by the question-answering model is processed to further improve the question-answering model's protection performance against harmful output.

[0050] It should be noted that the first harmful information and the second harmful information may be the same harmful information.

[0051] In some embodiments, after obtaining the text content output by the question-answering model, it also includes: filtering sensitive information in the text content output by the question-answering model. Specifically, sensitive information can be identified through pattern recognition (for example, unique patterns of ID card numbers and mobile phone numbers) and named entity recognition, and sensitive information in the text content output by the question-answering model can be removed to prevent privacy leakage.

[0052] In some embodiments, when it is detected that the text content output by the question-answering model contains second harmful information, it also includes: using the user input text to fine-tune the question-answering model, that is, re-inputting the user input text (the user input text corresponding to the output text content containing the second harmful information) as an adversarial sample into the question-answering model, and fine-tuning the question-answering model to further improve the protection performance of the question-answering model against harmful outputs.

[0053] Furthermore, when the question-answering model outputs textual content containing the second harmful information based on the user input text, a new security rule corresponding to the user input text is added to the security rule library to update the security rule library, and a new security prompt word corresponding to the new security rule is added to the security prompt word library. The new security rule is a new rule established by the user based on the user input text, and the new security prompt word is a textual description of the new security rule established by the user.

[0054] The following describes the large-model-based intelligent question-and-answer protection device provided by the present invention. The large-model-based intelligent question-and-answer protection device described below and the large-model-based intelligent question-and-answer protection method described above can refer to each other.

[0055] The intelligent question-answering protection device based on a large model in the embodiment of the present invention is as follows: Figure 2 As shown, it includes the following modules: The prompt word fusion module 210 is used to fuse the user input text and the security prompt word to obtain a fused prompt word, where the security prompt word is a description text describing the security rules defined in the security rule library.

[0056] The question-answering model execution module 220 is used to input the fusion prompt word into the question-answering model to obtain the text content output by the question-answering model, wherein the question-answering model is obtained through adversarial training based on a training set with adversarial samples, and the adversarial samples are sample texts containing sample harmful information and sample safety prompt words.

[0057] The large-model-based intelligent question-and-answer protection device of this embodiment fuses security prompt words with user input text to form fused prompt words, that is, the fused prompt words not only include the user's question, but also include security prompt words used to constrain the output of the question-and-answer model based on the large model, so that the question-and-answer model can answer questions in the user input text within the constraints of the security rules. Moreover, the question-and-answer model itself is obtained through adversarial training based on a training set with adversarial samples, so that it can effectively deal with user input text with complex logic, and effectively prevent the question-and-answer model from being induced by the user to output content containing harmful information, thereby achieving effective protection of the output content of the large-model question and answer.

[0058] In some embodiments, the prompt word fusion module 210 specifically includes: The risk level search module is used to determine the target security risk level of the user input text based on entity keywords in the user input text.

[0059] The security prompt word selection module is used to select a security prompt word that matches the target security risk level from a security prompt word library based on the target security risk level.

[0060] A fusion module is selected to fuse the security prompt word matching the target security risk level with the user input text to obtain the fused prompt word.

[0061] The security prompt word library stores a plurality of security risk prompt words corresponding to each of the security rules, and the plurality of security risk prompt words correspond to corresponding security risk levels, respectively.

[0062] In some embodiments, the risk level search module is specifically used to: identify multiple entity keywords in the user input text; search for each target risk word that matches each entity keyword in the mapping table of security risk levels and risk words; and determine the highest security risk level among the security risk levels corresponding to each target risk word as the target security risk level.

[0063] In some embodiments, the intelligent question-answering protection device based on a large model further includes: a sensitive information filtering module for filtering out sensitive information in the user input text before inputting the fusion prompt word into the question-answering model.

[0064] In some embodiments, the intelligent question-and-answer protection device based on a large model further includes: a text replacement module, which is used to check the first harmful information in the user input text before inputting the fusion prompt word into the question-and-answer model; when the first harmful information is checked, determine the replacement information corresponding to the first harmful information based on the replacement mapping table, and use the replacement information to replace the first harmful information to form a replacement text; and replace the user input text in the fusion prompt word with the replacement text.

[0065] In some embodiments, the intelligent question-and-answer protection device based on a large model also includes: an application field judgment module, which is used to identify multiple entity keywords in the user input text before fusing the user input text and the security prompt word to obtain the fused prompt word; when all entity keywords do not match the field keywords in the whitelist, it outputs that the user input text does not conform to the field scope of the question-and-answer model and replaces the prompt of the user input text, wherein the whitelist specifies multiple field keywords of the application field of the problem handled by the question-and-answer model.

[0066] In some embodiments, the intelligent question-and-answer protection device based on a large model further includes: an output text processing module, which is used to perform a security check on the text content output by the question-and-answer model based on a security check template library corresponding to the question-and-answer model after obtaining the text content output by the question-and-answer model; the security check template library defines the second harmful information and corresponding avoidance strategies in the application field processed by the question-and-answer model; when it is detected that the text content output by the question-and-answer model contains second harmful information, the second harmful information is processed based on the avoidance strategy.

[0067] In some embodiments, the intelligent question-answering protection device based on a large model further includes: a model fine-tuning module, which is used to fine-tune the question-answering model using the user input text when it is detected that the text content output by the question-answering model contains second harmful information.

[0068] Figure 3 An example of a physical structure diagram of an electronic device is shown below. Figure 3 As shown, the electronic device may include: a processor 310, a communications interface 320, a memory 330, and a communication bus 340, wherein the processor 310, the communications interface 320, and the memory 330 communicate with each other via the communication bus 340. The processor 310 may call the logic instructions in the memory 330 to execute the intelligent question-answering protection method based on the large model, which includes: Fusing the user input text and the security prompt word to obtain a fused prompt word, wherein the security prompt word is a description text describing the security rules defined in the security rule library; The fused prompt word is input into a question-answering model to obtain text content output by the question-answering model, wherein the question-answering model is obtained through adversarial training based on a training set with adversarial samples, and the adversarial samples are sample texts containing sample harmful information and sample safety prompt words.

[0069] Furthermore, the logic instructions in the aforementioned memory 330 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product, stored in a storage medium, includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0070] On the other hand, the present invention further provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can perform the large model-based intelligent question-answering protection method provided by the above methods, which includes: Fusing the user input text and the security prompt word to obtain a fused prompt word, wherein the security prompt word is a description text describing the security rules defined in the security rule library; The fused prompt word is input into a question-answering model to obtain text content output by the question-answering model, wherein the question-answering model is obtained through adversarial training based on a training set with adversarial samples, and the adversarial samples are sample texts containing sample harmful information and sample safety prompt words.

[0071] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the large-model-based intelligent question-answering protection method provided by the above methods, comprising: Fusing the user input text and the security prompt word to obtain a fused prompt word, wherein the security prompt word is a description text describing the security rules defined in the security rule library; The fused prompt word is input into a question-answering model to obtain text content output by the question-answering model, wherein the question-answering model is obtained through adversarial training based on a training set with adversarial samples, and the adversarial samples are sample texts containing sample harmful information and sample safety prompt words.

[0072] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0073] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0074] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A large-model-based intelligent question-answering protection method, characterized in that: include: Fusing the user input text and the security prompt word to obtain a fused prompt word, wherein the security prompt word is a description text describing the security rules defined in the security rule library; The fused prompt word is input into a question-answering model to obtain text content output by the question-answering model, wherein the question-answering model is obtained through adversarial training based on a training set with adversarial samples, and the adversarial samples are sample texts containing sample harmful information and sample safety prompt words.

2. The intelligent question-answering protection method based on a large model according to claim 1 is characterized in that: The user input text and the security prompt word are merged to obtain the fused prompt word, including: Determining a target security risk level of the user input text based on entity keywords in the user input text; Based on the target security risk level, selecting a security prompt word that matches the target security risk level from a security prompt word library; fusing the security prompt word that matches the target security risk level with the user input text to obtain the fused prompt word; The security prompt word library stores a plurality of security risk prompt words corresponding to each of the security rules, and the plurality of security risk prompt words correspond to corresponding security risk levels, respectively.

3. The intelligent question-answering protection method based on a large model according to claim 2 is characterized in that: Determining a target security risk level of the user input text based on entity keywords in the user input text includes: Identify multiple entity keywords in user input text; Searching for each target risk word that matches each entity keyword in a mapping table of security risk levels and risk words; The highest security risk level among the security risk levels corresponding to the target risk words is determined as the target security risk level.

4. The intelligent question-answering protection method based on a large model according to claim 1 is characterized in that: Before inputting the fused prompt word into the question-answering model, the method further includes: filtering out sensitive information in the user input text.

5. The intelligent question-answering protection method based on a large model according to claim 1 is characterized in that: Before inputting the fusion prompt word into the question-answering model, the method further includes: Checking the user input text for first harmful information; In the case where the first harmful information is detected, determining replacement information corresponding to the first harmful information based on a replacement mapping table, and replacing the first harmful information with the replacement information to form a replacement text; The user input text in the fusion prompt word is replaced with the replacement text.

6. The intelligent question-answering protection method based on a large model according to claim 1 is characterized in that: Before fusing the user input text and the security prompt word to obtain the fused prompt word, the following steps are also included: Identify multiple entity keywords in user input text; When all entity keywords do not match the domain keywords in the whitelist, the user input text is output as not meeting the domain scope of the question-answering model, and a prompt is given to replace the user input text, wherein the whitelist specifies multiple domain keywords of the application domain of the problem handled by the question-answering model.

7. The intelligent question-answering protection method based on a large model according to any one of claims 1 to 6, characterized in that: After obtaining the text content output by the question-answering model, the method further includes: Based on the security check template library corresponding to the question-answering model, a security check is performed on the text content output by the question-answering model; the security check template library defines the second harmful information in the application field processed by the question-answering model and the corresponding avoidance strategy; When it is detected that the text content output by the question-answering model contains second harmful information, the second harmful information is processed based on the avoidance strategy.

8. The intelligent question-answering protection method based on a large model according to claim 7 is characterized in that: In the case where it is detected that the text content output by the question-answering model contains second harmful information, the method further includes: fine-tuning and training the question-answering model using the user input text.

9. An intelligent question-answering protection device based on a large model, characterized in that: include: A prompt word fusion module is used to fuse the user input text and the security prompt word to obtain a fused prompt word, wherein the security prompt word is a description text describing the security rules defined in the security rule library; The question-answering model execution module is used to input the fusion prompt word into the question-answering model to obtain the text content output by the question-answering model, wherein the question-answering model is obtained through adversarial training based on a training set with adversarial samples, and the adversarial samples are sample texts containing sample harmful information and sample safety prompt words.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the large model-based intelligent question-answering protection method according to any one of claims 1 to 8 is implemented.

Images