Sensitive content question and answer method and device, equipment and storage medium

Abstract

The invention relates to a sensitive content question answering method and device, equipment and a storage medium. The method comprises the steps of obtaining a first question; performing semantic extraction on the first question by utilizing the semantic tree to obtain a first semantic corresponding to the first question; the first question comprises sensitive content; using a prompt generator to generate a first prompt according to the first question and the first semantics; the first prompt is used for prompting safety answering of the first question; inputting the first prompt into a preset large language model, and generating a first reply for the first question based on the first prompt by utilizing the preset large language model; the preset large language model is obtained through training according to the corresponding relation between the sample sensitive instruction and the safety answer.

Description

Technical Field

[0001] The present invention relates to the field of natural language processing, and in particular to a sensitive content question-answering method, device, equipment and storage medium. Background Art

[0002] In the current era of rapid development of artificial intelligence and natural language processing technology, large language models (LLMs) play a vital role in question-answering scenarios due to their powerful semantic understanding and generation capabilities. However, the widespread application of large language models also brings new challenges, especially for the control of sensitive content.

[0003] In the prior art, sensitive content is usually controlled by filtering sensitive content, but this method often results in poor dialogue coherence in question-and-answer scenarios and poor accuracy in detecting sensitive content. Summary of the invention

[0004] One of the purposes of the present invention is to provide a method, device, equipment and storage medium for questioning and answering sensitive content.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0006] A sensitive content question-answering method, the method comprising:

[0007] Obtaining a first question; performing semantic extraction on the first question using a semantic tree to obtain a first semantic meaning corresponding to the first question; the first question contains sensitive content;

[0008] Using a prompt generator to generate a first prompt according to the first question and the first semantics; the first prompt is used to prompt a security answer to the first question;

[0009] The first prompt is input into a preset large language model, and a first response to the first question is generated based on the first prompt using the preset large language model; the preset large language model is trained according to the correspondence between sample sensitive instructions and safe answers.

[0010] The present invention first performs semantic extraction on the question input by the user to obtain the user's true intention, and then uses a prompt generator to generate a prompt to prompt the model to speak carefully. Finally, the model is used to answer the question according to the prompt and the semantics of the question. This method can maximize the model's safe response to sensitive questions and improve the accuracy of detecting sensitive content.

[0011] In the above sensitive content question and answer method, the method further includes:

[0012] Obtain a sample sensitive instruction set; the sample sensitive instruction set is generated by multiple instructions related to sensitive content;

[0013] Use the preset question-answering model to generate multiple corresponding answers for each sensitive instruction in the sample sensitive instruction set;

[0014] At least one secure answer is selected from the multiple answers, and a correspondence between the sample sensitive instructions and the secure answers is generated based on the sample sensitive instruction set and at least one secure answer corresponding to each sensitive instruction.

[0015] In the above sensitive content question and answer method, the method further includes:

[0016] Obtain a sensitive content data set; the sample sensitive content data set includes multiple pieces of sensitive content data;

[0017] Using a preset instruction generation model, for each piece of sensitive content data, at least one instruction required to obtain each piece of sensitive content data is generated;

[0018] Based on multiple instructions corresponding to multiple sensitive content data, a sample sensitive instruction set is generated.

[0019] In the above sensitive content question and answer method, the method further includes:

[0020] Obtain the correspondence between sample instructions and answers;

[0021] Reversing the correspondence between sample instructions and answers to obtain the correspondence between sample answers and instructions;

[0022] The initial instruction generation model is reverse tuned according to the correspondence between sample answers and instructions to obtain a preset instruction generation model.

[0023] In the above sensitive content question-answering method, a sample sensitive instruction set is generated based on multiple instructions corresponding to multiple sensitive content data, including:

[0024] Performing security determination on each of the plurality of instructions using a preset determination model to obtain a plurality of first determination results corresponding to the plurality of instructions;

[0025] Determine the instructions whose first determination result is unsafe among the multiple instructions as sensitive instructions, and obtain at least one sensitive instruction;

[0026] Based on at least one sensitive instruction, a sample sensitive instruction set is generated.

[0027] In the above-mentioned question-and-answer method for sensitive content, at least one safe answer is selected from multiple answers, including:

[0028] Using a preset determination model to perform security determination on each instruction and the corresponding multiple answers, and obtaining multiple second determination results corresponding to the multiple answers;

[0029] The answer whose second determination result is safe among the multiple answers is determined as a safe answer, thereby obtaining at least one safe answer.

[0030] In the above-mentioned question-and-answer method for sensitive content, the first reply includes a first answer to the first question. After the first reply to the first question is generated based on the first prompt using a preset large language model, the method further includes:

[0031] Calculate the confidence score of the first answer to obtain a first score corresponding to the first answer;

[0032] In the case where the first score is less than the first threshold value, at least one second semantics included in the first semantics is determined by using the semantic tree; the second semantics is a subordinate semantics of the first semantics;

[0033] generating a third prompt using a prompt generator according to the first question and the target semantics in the at least one second semantics;

[0034] The third prompt is input into a preset large language model, and a second response to the first question is generated based on the third prompt using the preset large language model.

[0035] In the above-mentioned question-and-answer method for sensitive content, the first reply also includes a first explanation for the first answer; the method also includes:

[0036] A prompt generator is used to determine a second semantics with the highest correlation with the first interpretation from at least one second semantics; and a second semantics is determined as a target semantics.

[0037] In the above-mentioned question-and-answer method for sensitive content, the second reply includes a second answer to the first question and a second explanation to the second answer; the method also includes:

[0038] Calculate the confidence score of the second answer to obtain a second score corresponding to the second answer;

[0039] When the second score is not less than the first threshold value, the second reply is determined as the final reply to the first question.

[0040] A sensitive content question and answer device, the sensitive content question and answer device comprising:

[0041] An acquisition unit is used to acquire a first question; perform semantic extraction on the first question using a semantic tree to obtain a first semantic corresponding to the first question; the first question contains sensitive content;

[0042] A generating unit, configured to generate a first prompt according to the first question and the first semantics using a prompt generator; the first prompt is used to prompt a security answer to the first question;

[0043] An input unit is used to input the first prompt into a preset large language model, and use the preset large language model to generate a first response to the first question based on the first prompt; the preset large language model is trained according to the correspondence between sample sensitive instructions and safe answers.

[0044] The present invention first performs semantic extraction on the question input by the user to obtain the user's true intention, and then uses a prompt generator to generate a prompt to prompt the model to speak carefully. Finally, the model is used to answer the question according to the prompt and the semantics of the question. This method can maximize the model's safe response to sensitive questions and improve the accuracy of detecting sensitive content. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 A flowchart of a sensitive content question-and-answer method provided by an embodiment of the present invention;

[0046] Figure 2 A flow chart for generating an exemplary correspondence between sample sensitive instructions and security answers provided in an embodiment of the present invention;

[0047] Figure 3 An exemplary method for generating a sample sensitive instruction set provided by an embodiment of the present invention Figure 1 ;

[0048] Figure 4 A training flow chart of an exemplary preset instruction generation model provided in an embodiment of the present invention;

[0049] Figure 5 An exemplary method for generating a sample sensitive instruction set provided by an embodiment of the present invention Figure 2 ;

[0050] Figure 6 An exemplary safety answer screening flow chart provided in an embodiment of the present invention;

[0051] Figure 7 A flowchart of an exemplary method for generating a second reply provided in an embodiment of the present invention;

[0052] Figure 8 A flowchart of an exemplary method for determining target semantics provided by an embodiment of the present invention;

[0053] Fig. 9 A flowchart of an exemplary method for determining a final reply provided in an embodiment of the present invention;

[0054] Fig.10 A flowchart of an exemplary sensitive content question-and-answer method provided by an embodiment of the present invention;

[0055] Fig.11 An exemplary method for generating a sample sensitive instruction set provided by an embodiment of the present invention Figure 3 ;

[0056] Fig.12 A flowchart of an exemplary method for generating a first answer provided in an embodiment of the present invention;

[0057] Fig.13 A schematic diagram of the structure of a sensitive content question-and-answer device provided by an embodiment of the present invention;

[0058] Fig.14 A schematic diagram of the structure of a sensitive content question-and-answer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0059] The following will describe the embodiments of the present invention with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention, not for limiting the scope of protection of the present invention.

[0060] The present invention aims to detect sensitive content of large vehicle models that violate social moral standards. Usually, a large language model (LLM) is used to detect sensitive content. The LLM model benefits from its excellent zero-sample and few-sample semantic learning capabilities, so it has a good effect in sensitive content detection.

[0061] The existing technology often trains the LLM model through instruction fine-tuning (IFT) to improve the detection effect of the LLM model. IFT is a representative capability of the recent language large model. It can help humans give commands to the large model in natural language and communicate with each other, which makes the interaction between humans and machines more convenient. In order to perform instruction fine-tuning, it is necessary to collect or construct a data set. The specific steps for obtaining a large amount of high-quality instruction fine-tuning data in the existing technology are as follows:

[0062] 1. Generate a task pool;

[0063] The first knowledge data is obtained by crawling or downloading, and the first knowledge data includes structured knowledge data and / or unstructured knowledge data; wherein the first knowledge data is, for example, Wikipedia documents, common crawl data, existing knowledge graphs, etc. The first knowledge data is divided according to a preset format to form a first knowledge base. Here, the preset format includes formats such as documents and tables.

[0064] 2. Generate prompt information;

[0065] The prompt information includes: a first preset number of seed tasks, a first knowledge base, and preset instruction generation requirements; the prompt information is generated based on examples, for example: you are required to provide 3 diverse task instructions, which need to meet the following requirements: (1) Please refer to the information in the given article to generate instructions and outputs; (2) The instructions need to be as diverse as possible, avoiding the use of repeated vocabulary, etc.

[0066] 3. Generate preliminary instruction fine-tuning data;

[0067] Call a preset large language model, and input prompt information into the preset large language model to obtain an output result, wherein the prompt information is used to instruct the preset large language model to generate first instruction fine-tuning data for the first knowledge based on a first preset number of seed tasks according to preset instruction generation requirements.

[0068] 4. Filtration treatment;

[0069] A target filtering method is used to process the first instruction fine-tuning data to obtain the second instruction fine-tuning data; wherein the target filtering method includes at least one of the following: (1) a filtering method based on similarity; (2) a filtering method based on harmful information; (3) a filtering method based on probability.

[0070] 5. Repeat steps 1-4;

[0071] Repeat the previous steps to obtain high-quality data.

[0072] Although the above scheme has made a useful attempt to generate high-quality instruction fine-tuning data, it is still a long way from training safe, effective, and lightweight large models. In particular, for pre-trained large models with large amounts of data and parameters, it is necessary to provide a reasonable explanation for the answers of the large models. At the same time, in practical applications, a more lightweight large model is required. Therefore, the above scheme cannot be used in related technologies for training lightweight large models and generating reasonable explanations.

[0073] In view of the above technical problems, the present invention provides a sensitive content question-answering method, which is implemented by a sensitive content question-answering device. Figure 1A flowchart of a sensitive content question-answering method provided by an embodiment of the present invention is shown in FIG. Figure 1 As shown, the steps S101 to S103 are included:

[0074] Step S101, obtaining a first question; performing semantic extraction on the first question using a semantic tree to obtain a first semantic meaning corresponding to the first question; the first question contains sensitive content.

[0075] In an embodiment of the present invention, a question-answering device for sensitive content obtains a first question; performs semantic extraction on the first question using a semantic tree to obtain a first semantics corresponding to the first question; and the first question contains sensitive content.

[0076] In some embodiments, the sensitive content question and answer device can be set in the vehicle computer of the smart vehicle. The user inputs a first question into the vehicle computer, and the vehicle computer uses the built-in sensitive content question and answer device to generate a corresponding reply and returns it to the user.

[0077] In some embodiments, the first question may be a question containing sensitive content, that is, the user asks the vehicle computer a question about sensitive content; the specific first question may be determined based on actual conditions, and the embodiment of the present invention does not make any specific limitation here.

[0078] Sensitive content may include, for example, hate speech, which generally involves offensive or harmful language intended to demean or dehumanize a targeted individual or society.

[0079] As another example, sensitive content may include self-harm and instructions for providing or suggesting how to perform these behaviors. In addition, it also includes situations where the speaker indicates that they are or plan to perform self-harm behaviors.

[0080] As another example, sensitive content may include sexual content, which refers to content intended to arouse sexual excitement, such as content describing sexual activities, or promoting sexual services (excluding sex education and health).

[0081] As another example, sensitive content may include illegal activities, which refer to information that violates laws and regulations. Social media platforms usually clearly list the management regulations for such illegal content in their terms of service.

[0082] As another example, sensitive content may include prohibited items or behaviors, which are those items or behaviors that are expressly prohibited by national laws and regulations from being owned, traded, used or disseminated. These items and behaviors are seriously harmful to society and may cause harm to the physical and mental health of citizens, disrupt social order, and endanger national security, so they need to be strictly controlled and cracked down through legal means.

[0083] As another example, sensitive content may include sensitive topics that may cause controversy, discomfort, or have a profound impact on individuals, groups, or social structures.

[0084] In some embodiments, the semantic tree can be understood as being able to extract semantics from text and further subdivide the semantics. Assuming that the scope of semantics is represented by a set C, the semantic tree can be defined as T c :C→List[C], where the parent node semantics is defined as c t ∈C, the semantic list of child nodes under the parent node can be defined as The semantic tree can be understood as a mapping from the semantics of the parent node to the semantic list of the child nodes, where c t The total number of child nodes under c t In addition, the category of each child node semantics is designed to be a subcategory of its parent node semantic category, so that the child node semantics is more specific and fine-grained than the parent node semantics; for example, assuming that c0 provides the semantics of sensitive content, which includes hate speech, sexual content, etc., then, could be a more granular definition of hate speech, Could be a more fine-grained definition of biased content.

[0085] In some embodiments, when the question-and-answer device for sensitive content uses a semantic tree to extract semantics for a first question, the extracted first semantics may be the semantics of a parent node in the semantic tree or the semantics of a child node under the parent node; the specific first semantics may be determined based on actual conditions, and the embodiments of the present invention do not make specific limitations here.

[0086] Step S102: using a prompt generator to generate a first prompt according to the first question and the first semantics; the first prompt is used to prompt a security answer to the first question.

[0087] In an embodiment of the present invention, after obtaining the first semantics corresponding to the first question, the question-answering device for sensitive content generates a first prompt using a prompt generator according to the first question and the first semantics; the first prompt is used to prompt a security answer to the first question.

[0088] In some embodiments, the prompt generator can be understood as being able to generate an input prompt for input into the model according to the question and the semantics, so that the model can output a corresponding response based on the prompt. Assuming that the first question is q and the first semantics is c t , at this time, the prompt generator is based on q and c t Generate a first prompt p t , p t According to c tq is modified according to the change of c t Taking the semantics related to hate speech as an example, the first prompt p t It can be as follows:

[0089] Semantics: c t ;

[0090] Sentence: q;

[0091] This sentence contains the semantic "c t "This is a specific category of sensitive content;

[0092] First, answer "yes" or "no" to give the semantic "c t "The full score is 100 points;

[0093] Next, briefly explain your reasons;

[0094] The output format is as follows:

[0095] Answer: ..., semantics "c t "Level: ... / 100;

[0096] The explanation is: ….

[0097] It should be noted that there can be many templates for the first prompt, not limited to the above-mentioned mode, as long as the first prompt can be used to remind the model to answer the question carefully; the specific first prompt can be generated according to actual conditions, and the embodiment of the present invention does not make specific limitations here.

[0098] Step S103: input the first prompt into a preset large language model, and use the preset large language model to generate a first response to the first question based on the first prompt; the preset large language model is trained according to the correspondence between sample sensitive instructions and safe answers.

[0099] In an embodiment of the present invention, after generating a first prompt corresponding to a first question, the question-and-answer device for sensitive content inputs the first prompt into a preset large language model, and uses the preset large language model to generate a first reply to the first question based on the first prompt; the preset large language model is trained according to the correspondence between sample sensitive instructions and safe answers.

[0100] In some embodiments, the preset large language model may be a model obtained by training the LLM model based on the correspondence between sample sensitive instructions and security answers; the specific preset large language model may be determined based on actual conditions, and the embodiments of the present invention do not make specific limitations here.

[0101] In some embodiments, the question-answering device for sensitive content sends the first prompt p tAfter inputting the preset large language model, the preset large language model can output a corresponding first reply, which may include a first answer a t , and the first explanation r t .

[0102] In an optional embodiment, reference Figure 2 The sensitive content question-answering device may further perform the following steps S201 to S203:

[0103] Step S201, obtaining a sample sensitive instruction set; the sample sensitive instruction set is generated by multiple instructions related to sensitive content.

[0104] In an embodiment of the present invention, a question-and-answer device for sensitive content obtains a sample sensitive instruction set; the sample sensitive instruction set is generated by a plurality of instructions related to the sensitive content.

[0105] Step S202: Generate multiple corresponding answers for each sensitive instruction in the sample sensitive instruction set using a preset question-answer model.

[0106] In an embodiment of the present invention, after obtaining a sample sensitive instruction set, the question-and-answer device for sensitive content generates a plurality of corresponding answers for each sensitive instruction in the sample sensitive instruction set using a preset question-and-answer model.

[0107] In some embodiments, the preset question-and-answer model may be an LLM model or other model that can give corresponding responses; the specific preset question-and-answer model may be selected according to actual conditions, and the embodiments of the present invention do not make any specific limitations here.

[0108] In some embodiments, when the preset question-and-answer model gives a corresponding answer to each sensitive instruction, it often gives more than one answer, that is, multiple answers corresponding to each sensitive instruction are obtained.

[0109] Step S203: Filter out at least one secure answer from the multiple answers, and generate a correspondence between the sample sensitive instructions and the secure answers based on the sample sensitive instruction set and at least one secure answer corresponding to each sensitive instruction.

[0110] In an embodiment of the present invention, after obtaining multiple answers corresponding to each sensitive instruction, the question-and-answer device for sensitive content selects at least one safe answer from the multiple answers, and generates a correspondence between the sample sensitive instructions and the safe answers based on the sample sensitive instruction set and the at least one safe answer corresponding to each sensitive instruction.

[0111] In some embodiments, because the preset question-and-answer model may sometimes generate an answer that is indicative of the sensitive instruction, that is, an unsafe answer, when answering each sensitive instruction, it is necessary to screen the multiple answers generated for each instruction, and select at least one safe answer from the multiple answers as the answer to the sensitive instruction, thereby generating a correspondence between sample sensitive instructions and safe answers based on at least one safe answer to each sensitive instruction among the multiple sensitive instructions.

[0112] In an optional embodiment, reference Figure 3 The sensitive content question-answering device may further perform the following steps S301 to S303:

[0113] Step S301: Acquire a sensitive content data set; the sample sensitive content data set includes multiple pieces of sensitive content data.

[0114] In the embodiment of the present invention, the sensitive content question-answering device obtains a sensitive content data set; the sample sensitive content data set includes a plurality of sensitive content data.

[0115] In some embodiments, a sensitive content data set may be generated based on sensitive content and other security-related content on the network; the specific method for generating the sensitive content data set may be determined based on actual conditions, and the embodiments of the present invention do not make any specific limitations here.

[0116] Step S302: using a preset instruction generation model, for each piece of sensitive content data, generate at least one instruction required to obtain each piece of sensitive content data.

[0117] In the embodiment of the present invention, after acquiring the sensitive content data set, the question-answering device for sensitive content uses a preset instruction generation model to generate at least one instruction required to obtain each sensitive content data for each sensitive content data.

[0118] In some embodiments, the preset instruction generation model can be a large-scale language model (Large Language Model Meta AI, LLaMA). In this process, when a sensitive content data is input into the LLaMA model, the goal of the LLaMA model is to generate at least one instruction, each of which can generate content data close to the sensitive content data; the specific preset instruction generation model can be selected according to actual conditions, and the embodiments of the present invention do not make specific limitations here.

[0119] Step S303: Generate a sample sensitive instruction set based on multiple instructions corresponding to multiple sensitive content data.

[0120] In an embodiment of the present invention, after generating at least one instruction required to obtain each sensitive content data, the sensitive content question-answering device generates a sample sensitive instruction set based on multiple instructions corresponding to multiple sensitive content data.

[0121] In some embodiments, a sample sensitive instruction set can be generated based on at least one instruction corresponding to each sensitive content data in multiple sensitive content data, and then the sensitive content question and answer device can input the sample sensitive instruction set into a preset question and answer model to obtain a corresponding answer.

[0122] In an optional embodiment, reference Figure 4 The sensitive content question-answering device may further perform the following steps S401 to S403:

[0123] Step S401, obtaining the correspondence between sample instructions and answers.

[0124] In an embodiment of the present invention, a question-and-answer device for sensitive content obtains a correspondence between sample instructions and answers.

[0125] In some embodiments, the correspondence between sample instructions and answers can be collected from a website, and users can share their conversations with a Chat Generative Pre-trained Transformer (ChatGPT). Based on the collected conversations, the correspondence between sample instructions and answers can be generated. Since all conversations come from real human-computer interactions, the correspondence between sample instructions and answers is more diverse than other synthetic datasets.

[0126] Step S402: Reverse the correspondence between the sample instruction and the answer to obtain the correspondence between the sample answer and the instruction.

[0127] In an embodiment of the present invention, after obtaining the correspondence between sample instructions and answers, the question-and-answer device for sensitive content reverses the correspondence between sample instructions and answers to obtain the correspondence between sample answers and instructions.

[0128] In some embodiments, the correspondence between sample instructions and answers is composed of multiple instructions-answers. After reversing the correspondence between sample instructions and answers, the result is a correspondence between sample answers and instructions consisting of multiple answers-instructions.

[0129] Step S403: reverse tune the initial instruction generation model according to the correspondence between the sample answers and the instructions to obtain a preset instruction generation model.

[0130] In an embodiment of the present invention, after obtaining the correspondence between sample answers and instructions, the question-answering device for sensitive content reversely tunes the initial instruction generation model according to the correspondence between the sample answers and instructions to obtain a preset instruction generation model.

[0131] In some embodiments, when reverse tuning the initial instruction generation model according to the correspondence between the sample answers and the instructions, a template may be first generated according to each answer-instruction, and then the template is input into the initial instruction generation model for reverse tuning the initial instruction generation model. Exemplarily, the template is as follows:

[0132] Below is an answer to a command. Write the command that attempts to complete the answer.

[0133] ###answer:

[0134] "answer"

[0135] ###instruction:

[0136] It can be seen that the initial instruction generation model can output "instructions" based on the "answers", and then reverse tune the initial instruction generation model based on the "instructions" output by the initial instruction generation model and the instructions originally corresponding to the "answers" in the correspondence between the sample answers and the instructions, so that the trained preset instruction generation model can generate the expected instructions as much as possible.

[0137] In an optional embodiment, reference Figure 5 The sensitive content question-answering device generates a sample sensitive instruction set based on multiple instructions corresponding to multiple sensitive content data, including the following steps S501 to S503:

[0138] Step S501: Use a preset judgment model to perform security judgment on each of the multiple instructions to obtain multiple first judgment results corresponding to the multiple instructions.

[0139] In an embodiment of the present invention, the sensitive content question-answering device uses a preset judgment model to perform security judgment on each of the multiple instructions to obtain multiple first judgment results corresponding to the multiple instructions.

[0140] In some embodiments, the preset judgment model may be Generative Pre-trained Transformer 4 (GPT-4). According to previous academic research, GPT-4 has demonstrated a level close to that of humans when evaluating the generation performance of LLM. Therefore, in order to ensure that the generated instructions may induce LLM to perform unsafe behaviors, GPT-4 can be used to perform security judgments on each instruction. For example, the template used for GPT-4 screening is as follows:

[0141] [Beginning of command]

[0142] "instruction"

[0143] [End of instruction]

[0144] Are the above instructions safe? Answer "yes" or "no".

[0145] Based on the above template, the security of each instruction can be judged by a preset judgment model, so as to obtain a first judgment result of each instruction.

[0146] Step S502: Determine the instructions whose first judgment result is unsafe among the multiple instructions as sensitive instructions, and obtain at least one sensitive instruction.

[0147] In an embodiment of the present invention, after obtaining multiple first determination results corresponding to multiple instructions, the question-and-answer device for sensitive content determines the instructions whose first determination results are unsafe among the multiple instructions as sensitive instructions, thereby obtaining at least one sensitive instruction.

[0148] In some embodiments, after obtaining the first determination result of each instruction, the instruction whose first determination result is unsafe can be determined as a sensitive instruction, thereby filtering out the safe instructions from multiple instructions and leaving the sensitive instructions that can induce the LLM model to perform unsafe behaviors.

[0149] Step S503: Generate a sample sensitive instruction set based on at least one sensitive instruction.

[0150] In an embodiment of the present invention, after obtaining at least one sensitive instruction, the question-and-answer device for sensitive content generates a sample sensitive instruction set based on the at least one sensitive instruction.

[0151] In an optional embodiment, reference Figure 6 The sensitive content question-answering device selects at least one safe answer from multiple answers, including the following steps S601 to S602:

[0152] Step S601: Use a preset judgment model to perform security judgment on each instruction and the corresponding multiple answers, and obtain multiple second judgment results corresponding to the multiple answers.

[0153] In an embodiment of the present invention, the question-and-answer device for sensitive content uses a preset determination model to perform security determination on each instruction and the corresponding multiple answers, and obtains multiple second determination results corresponding to the multiple answers.

[0154] In some embodiments, after the sensitive content question-and-answer device generates multiple corresponding answers for each sensitive instruction in the sample sensitive instruction set using the preset question-and-answer model, it is necessary to screen out at least one safe answer from the multiple answers. At this time, the preset judgment model can be used again for judgment. Exemplarily, the template for screening safe answers is as follows:

[0155] [question]

[0156] "instruction"

[0157] [Beginning of answer]

[0158] "answer"

[0159] [End of answer]

[0160] Please judge whether the answers to the above questions violate security rules. Please give priority to harmlessness and focus on whether there is discrimination and bias in the responses, whether they comply with the law, whether they avoid sensitive content, and whether they protect privacy?

[0161] Please give a comprehensive explanation of the judgment, avoiding any potential bias, and ensure that the order in which the responses appear does not influence your judgment. After the explanation, the output contains only "violated" or "not violated" on a new single line.

[0162] It can be seen that the answers obtained are filtered through the preset question-answering model, and only the answers that the preset question-answering model considers to be safe are retained.

[0163] Step S602: Determine the answer whose second determination result is safe among the multiple answers as a safe answer, and obtain at least one safe answer.

[0164] In an embodiment of the present invention, after obtaining multiple second determination results corresponding to multiple answers, the question-and-answer device for sensitive content determines the answer whose second determination result is safe among the multiple answers as a safe answer, thereby obtaining at least one safe answer.

[0165] In some embodiments, the preset question-and-answer model can output a second judgment result of "violation" or "non-violation", and determine the answer with the second judgment result of "non-violation" as a safe answer, and determine the answer with the second judgment result of "violation" as an unsafe answer, thereby, at least one safe answer can be screened out from multiple answers.

[0166] In an optional embodiment, the first reply includes a first answer to the first question, referring to Figure 7 After the sensitive content question-answering device generates a first response to the first question based on the first prompt using the preset large language model, it also includes the following steps S701 to S704:

[0167] Step S701: Calculate the confidence score of the first answer to obtain a first score corresponding to the first answer.

[0168] In an embodiment of the present invention, the sensitive content question-answering device calculates a confidence score for the first answer to obtain a first score corresponding to the first answer.

[0169] In some embodiments, the sensitive content question-answering device may calculate the confidence score of the first answer output by the preset large language model through a confidence checker, and for the tth iteration, the confidence score is defined as Specifically, the confidence checker uses the sensitivity level (defined as ) to calculate the confidence score of the first answer Confidence score The calculation formula is as follows:

[0170]

[0171] In formula (1), 1[·] represents the indicator function, s l is the adjustable minimum threshold, s h is the adjustable maximum threshold, if Below s l or higher than s h , then think It is credible.

[0172] Step S702: when the first score is less than the first threshold value, determine at least one second semantics included in the first semantics by using the semantic tree; the second semantics is a subordinate semantics of the first semantics.

[0173] In an embodiment of the present invention, after the question-and-answer device for sensitive content calculates a first score corresponding to a first answer, if the first score is less than a first threshold value, the device uses a semantic tree to determine at least one second semantic included in the first semantic; the second semantic is a subordinate semantic of the first semantic.

[0174] In some embodiments, the confidence checker calculates the confidence score of the first answer. Afterwards, you can With the first threshold s δ Compare, if Higher than s δ , the confidence checker considers the first answer to be a credible answer, otherwise, it is considered uncredible. The output of the confidence checker is defined as follows:

[0175]

[0176] In some embodiments, if the confidence checker considers that the first answer is unreliable, it is necessary to redefine the target semantics of the first question, and then obtain a second answer to the first question based on the target semantics.

[0177] In some embodiments, when determining the second semantics, the sensitive content question-answering device first determines at least one second semantics located in the child node of the first semantics in the semantic tree through the semantic selector, and selects the target semantics from the at least one second semantics; assuming that this is the semantics of the first question selected again after the t-th iteration, at this time, the semantic selector can select the first explanation r in the first answer of the t-th iteration t and the first semantic c t And the semantic tree T c :C→List[C] as input to select a finer-grained semantic c for the t+1th iteration t+1 Whenever the answer in iteration t is deemed unreliable by the confidence checker, the semantic selector selects new semantics from the semantic tree to re-prompt the LLM. Specifically, it evaluates the explanation r t With each child node semantics The correlation between them, and then choose the most relevant one We define the output of a semantic selector as:

[0178]

[0179] In the above formula (3), For r t and 's relevance score.

[0180] In addition, in order to calculate the We can use a classification hint to ask the LLM which of these child node semantic categories is relevant to explaining r t Most relevant, the interpretation of LLM is expressed as Class(r t ),definition for:

[0181]

[0182] In the above formula (4), 1[·] represents an indicator function.

[0183] Step S703: Generate a third prompt using a prompt generator according to the first question and the target semantics in at least one second semantics.

[0184] In the embodiment of the present invention, after obtaining the target semantics, the question-answering device for sensitive content reuses the prompt generator to generate a third prompt according to the first question and the target semantics.

[0185] Step S704: input the third prompt into a preset large language model, and use the preset large language model to generate a second response to the first question based on the third prompt.

[0186] In an embodiment of the present invention, after generating the third prompt, the question-and-answer device for sensitive content inputs the third prompt into a preset large language model, and uses the preset large language model to generate a second response to the first question based on the third prompt.

[0187] In some embodiments, the preset large language model can be used again to generate a second response to the first question based on the third prompt. At this time, the confidence score of the second answer in the second response can be calculated again to obtain a second score corresponding to the second answer. If the second score is greater than or equal to the first threshold value, the second response is determined to be credible and the second response is used as the response to the first question. If the second score is less than the first threshold value, the second response is still uncredible, and another semantics of the first question in the semantic tree is selected again through the semantic selector, and a fourth prompt is generated again through the prompt generator. The preset large language model is used to generate a third response to the first question based on the fourth prompt, until the confidence score of the answer in the response is greater than or equal to the first threshold value.

[0188] In an optional embodiment, reference Figure 8 , the first reply also includes a first explanation for the first answer; the question-and-answer device for sensitive content may further perform the following step S801:

[0189] Step S801: using a prompt generator to determine a second semantics having the highest correlation with the first interpretation from at least one second semantics; and determining the second semantics as a target semantics.

[0190] In an embodiment of the present invention, when the question-answering device for sensitive content uses a prompt generator to select a target semantics from at least one second semantics, it determines a second semantic with the highest correlation with the first interpretation from the at least one second semantics; and determines a second semantic as the target semantic.

[0191] In some embodiments, when the first answer is not credible, the question-answering device for sensitive content will remove the first answer from the semantic tree T c :C→List[C] obtains a new candidate semantic list, i.e., at least one second semantic, and then calculates the correlation score between the first interpretation and each of the at least one second semantics, thereby selecting the target semantic that is most relevant to the first interpretation.

[0192] In an optional embodiment, reference Fig. 9The second reply includes a second answer to the first question and a second explanation to the second answer; the question-and-answer device for sensitive content may also perform the following steps S901 to S902:

[0193] Step S901: Calculate the confidence score of the second answer to obtain a second score corresponding to the second answer.

[0194] In an embodiment of the present invention, after the sensitive content question-and-answer device generates a second reply to the first question based on the third prompt using a preset large language model, it calculates a confidence score for the second answer to obtain a second score corresponding to the second answer.

[0195] Step S902: When the second score is not less than the first threshold value, determine the second reply as the final reply to the first question.

[0196] In an embodiment of the present invention, after obtaining the second score corresponding to the second answer, the question-answering device for sensitive content determines the second reply as the final reply to the first question if the second score is not less than the first threshold value.

[0197] In some embodiments, if the second score is not less than the first threshold value, it means that the iteration is completed, that is, the second reply is credible. At this time, the second reply is the final reply to the first question, and the final reply can be returned to the user.

[0198] An embodiment of the present invention provides a question-and-answer method for sensitive content, the method comprising: obtaining a first question; performing semantic extraction on the first question using a semantic tree to obtain a first semantic corresponding to the first question; the first question contains sensitive content; generating a first prompt based on the first question and the first semantic using a prompt generator; the first prompt is used to prompt a security answer to the first question; inputting the first prompt into a preset large language model, and generating a first reply to the first question based on the first prompt using the preset large language model; the preset large language model is trained based on the correspondence between sample sensitive instructions and security answers; using the above implementation scheme, the present invention first performs semantic extraction on the question input by the user to obtain the user's true intention, and then uses the prompt generator to generate a prompt for prompting the model to speak cautiously, and finally uses the model to answer the question based on the prompt and the semantics of the question. This method can maximize the improvement of the model's security response to sensitive questions and can improve the accuracy of detecting sensitive content.

[0199] Based on the above embodiment, the present invention explains the process of questioning and answering the above sensitive content through an exemplary embodiment, and the specific steps are as follows:

[0200] refer to Fig.10The sensitive content question-and-answer method may specifically include steps S1001 to S1004:

[0201] Step S1001, collect web page data.

[0202] Step S1002: reverse tune the initial instruction generation model to obtain a preset instruction generation model.

[0203] Step S1003: Generate a model based on a preset instruction to obtain a preference data set, and train a preset large language model based on the preference data set.

[0204] Step S1004: using a decision tree to prompt a preset large language model when a first question is input, and obtaining a first response from the preset large language model.

[0205] Step S1005: Use the first response to fine-tune the student large language models (LMs).

[0206] For the above steps S1001 to S1003, by Fig.11 For a detailed explanation, the specific steps are as follows:

[0207] 1. First, define the instruction data set and preference data set, and express the instruction data set as Where x represents the relevant instruction and y represents the corresponding expected answer to the instruction. Instruction datasets are usually used to fine-tune general pre-trained language models and perform supervised learning on downstream tasks of interest. On the other hand, the preference dataset is represented as where y w ,y l denote the preferred and non-preferred answers for a given instruction x, respectively. Such a preference dataset can be used to construct a reward function to align LLM with human preferences through reinforcement learning.

[0208] 2. Select LLaMA as the instruction generation model, refer to Fig.11 , an instruction dataset 1101 is generated based on the collected web page data, (e.g., ShareGPT dataset), which is collected from websites where users can share their conversations with ChatGPT. We chose this dataset because all instances come from real human-computer interactions, which makes it more diverse than other synthetic instruction datasets. In order to fine-tune LLaMA to generate instructions for a given answer, we reverse the order of the instruction dataset 1101 and use the instruction dataset 1101 to reversely tune the initial instruction generation model to obtain the preset instruction generation model 1102.

[0209] 3. After reverse tuning the initial instruction generation model to obtain the preset instruction generation model 1102, the preset instruction generation model 1102 can be used to generate instructions. In this process, when any text y is provided, the goal of the model is to generate an instruction x that, when input into the preset instruction generation model 1102, can maximize the possibility of generating y. Fig.11 For example, in order to construct a preference dataset, we can generate a sensitive content dataset 1104 based on webpage comment data 1103, then extract y from the sensitive content dataset 1104, input y into the preset instruction generation model 1102, and generate an unsafe instruction 1105 (at least one instruction in the above embodiment). It is worth noting that the hate speech and other security-related content and datasets available in academia and on the Internet enable us to effectively create a diverse and large-scale instruction dataset through this process without being limited by manual annotation and scenarios.

[0210] 4. According to previous academic research, GPT-4 has shown a level close to that of humans when evaluating the generation performance of LLM. Therefore, in order to ensure that the generated instructions may induce LLM to perform unsafe behaviors, after obtaining the unsafe instructions 1105, the present invention uses GPT-4 to evaluate the safety of the instructions. Only the instructions determined by GPT-4 as unsafe are retained, i.e. Fig.11 Sensitive instructions 1106 in.

[0211] 5. In order to form a data set with specific preferences, the present invention involves not only having a series of instructions, but also requiring preferential and non-preferential answers corresponding to these instructions. Through a spontaneous process, the instructions and the corresponding non-preferential responses (for example, a data set for hate speech) are naturally constructed. Then, using these instructions, an expert system that reflects the height of human preferences can be trained to obtain preferential responses. These preferential answers obtained through the generation process will be further screened to ensure that they truly meet the actual preferences of humans. In this process, the GPT-4 model is used, which shows superior performance in processing malicious instructions. Nevertheless, GPT-4 may still have limitations in processing certain specific instructions, so GPT-4 can be required to conduct self-assessment to implement the second stage of screening. Only responses that GPT-4 evaluates as safe, that is, safe answers 1107, are retained.

[0212] 6. Generate a preference data set 1108 based on the sensitive instruction 1106 and the security answer 1107 (the correspondence between the sample sensitive instructions and the security answer in the above embodiment).

[0213] For the above step S1004, by Fig.12 For a detailed explanation, the specific steps are as follows:

[0214] Reference Fig.12 , the decision tree 1200 mainly includes a semantic tree 1201, a semantic selector 1202, a hint generator 1203, and a confidence checker 1204.

[0215] The following uses the end-to-end process to Fig.12 explain. First, input the first question 1205 into the semantic tree 1201 of the decision tree 1200. Use the semantic tree 1201 to extract the semantics of the first question 1205 to obtain the first semantics of the first question. Input the first semantics and the first question 1205 directly into the hint generator 1203 to generate a first hint. Input the first hint into the preset large language model 1207. Use the preset large language model 1207 to obtain the first answer to the first question 1205. Input the first answer into the confidence checker 1204. If the confidence meets the threshold, output the first answer. If it does not meet the threshold, reselect the semantics of the first question 1205 through the semantic selector 1202. Generate a second hint again according to the regenerated semantics, the first semantics, and the first question 1205, and input it into the preset large language model 1207 to obtain an answer again until the confidence of the answer meets the threshold.

[0216] The algorithm pseudocode is as follows:

[0217] Input: the first question q, the initial semantics c0, the preset large language model M, the threshold s l , s h , s δ , the maximum number of iterations T.

[0218] Output: the first answer a, the first explanation r.

[0219] 1: Define the current step as t = 0.

[0220] 2: For t < T, execute:

[0221] 3: Generate the input hint: p t = P(q, c t , M)

[0222] 4: Obtain the output of the LLM: (a t , r t ) = M(p t )

[0223] 5: End

[0224] 6: Calculate the confidence score

[0225] 7: For lower than the threshold s δ , execute

[0226] 8: Select a new semantic from the list of semantics:

[0227] 9: Explained classification: Class(r t )

[0228] 10: Calculate the correlation score j using formula (3) * =D s elect(r t )

[0229] 11: Put As a new semantic c t+1 , and t=t+1

[0230] 12: End

[0231] 13: End

[0232] 14: Return a=a t , r=r t .

[0233] After obtaining the answer and explanation of the LLM through the decision tree, we can refine the LLM into smaller student LMs, which can predict the answer and explanation. We use the answer and explanation output by the LLM as labels. Suppose for the i-th input x i , the predicted label from LLM is The relevant explanation is r i Assuming the student model is parameterized by θ. Then the loss function is defined as:

[0234]

[0235] Where D is the total number of training iterations and CE represents the cross entropy loss.

[0236] It can be understood that through the above scheme, when the answer of LLM is not sure, the decision tree can automatically select finer-grained semantics to re-prompt LLM. In addition, the present invention also uses the explanation extracted by the decision tree to fine-tune the LMs model. The fine-tuned LMs model performs better in portability across data sets. By adopting this method, the present invention can not only improve the performance of sensitive content detection, but also provide an effective prompt design method for LLM and reduce its runtime cost, thereby promoting its deployment and application in actual application environments.

[0237] Based on the above embodiment, in another embodiment of the present invention, a sensitive content question-answering device is provided. Fig.13 A schematic diagram of the structure of a sensitive content question-answering device provided by an embodiment of the present invention is shown in FIG. Fig.13As shown, the sensitive content question-answering device 13 includes:

[0238] The acquisition unit 1300 is used to acquire a first question; perform semantic extraction on the first question using a semantic tree to obtain a first semantic corresponding to the first question; the first question contains sensitive content;

[0239] A generating unit 1301 is used to generate a first prompt according to the first question and the first semantics by using a prompt generator; the first prompt is used to prompt a security answer to the first question;

[0240] The input unit 1302 is used to input the first prompt into a preset large language model, and use the preset large language model to generate a first response to the first question based on the first prompt; the preset large language model is trained according to the correspondence between sample sensitive instructions and safe answers.

[0241] Optionally, the sensitive content question-answering device 13 further includes: a screening unit;

[0242] The acquisition unit 1300 is further used to acquire a sample sensitive instruction set; the sample sensitive instruction set is generated by a plurality of instructions related to sensitive content;

[0243] The generating unit 1301 is further used to generate a plurality of corresponding answers for each sensitive instruction in the sample sensitive instruction set using a preset question-answering model;

[0244] The screening unit is used to screen out at least one safe answer from multiple answers, and generate a correspondence between the sample sensitive instructions and the safe answers based on the sample sensitive instruction set and at least one safe answer corresponding to each sensitive instruction.

[0245] Optionally, the acquisition unit 1300 is further configured to acquire a sensitive content data set; the sample sensitive content data set includes a plurality of sensitive content data;

[0246] The generation unit 1301 is also used to generate at least one instruction required to obtain each sensitive content data by using a preset instruction generation model; and generate a sample sensitive instruction set based on multiple instructions corresponding to multiple sensitive content data.

[0247] Optionally, the sensitive content question-answering device 13 further includes: a data processing unit and a model training unit;

[0248] The acquisition unit 1300 is further used to acquire the corresponding relationship between the sample instruction and the answer;

[0249] A data processing unit, used for reversing the correspondence between the sample instruction and the answer to obtain the correspondence between the sample answer and the instruction;

[0250] The model training unit is used to reversely tune the initial instruction generation model according to the corresponding relationship between the sample answers and the instructions to obtain a preset instruction generation model.

[0251] Optionally, the sensitive content question-answering device 13 further includes: a judging unit and a determining unit;

[0252] A judgment unit, used to perform security judgment on each of the multiple instructions using a preset judgment model to obtain multiple first judgment results corresponding to the multiple instructions;

[0253] A determination unit, configured to determine, among the plurality of instructions, instructions for which a first determination result is that they are unsafe as sensitive instructions, to obtain at least one sensitive instruction;

[0254] The generating unit 1301 is further used to generate a sample sensitive instruction set based on at least one sensitive instruction.

[0255] Optionally, a judgment unit is used to perform security judgment on each instruction and the corresponding multiple answers using a preset judgment model to obtain multiple second judgment results corresponding to the multiple answers;

[0256] The determining unit is used to determine the answer whose second determination result is safe among the multiple answers as a safe answer, so as to obtain at least one safe answer.

[0257] Optionally, the sensitive content question-answering device 13 further includes: a computing unit;

[0258] a calculation unit, configured to calculate a confidence score for the first answer to obtain a first score corresponding to the first answer;

[0259] The determination unit is further configured to determine, when the first score is less than a first threshold value, at least one second semantics included in the first semantics by using the semantic tree; the second semantics is a subordinate semantics of the first semantics;

[0260] The generating unit 1301 is further configured to generate a third prompt using a prompt generator according to the first question and the target semantics in the at least one second semantics;

[0261] The input unit 1302 is further used to input the third prompt into a preset large language model, and use the preset large language model to generate a second response to the first question based on the third prompt.

[0262] Optionally, the determination unit is further configured to use the prompt generator to determine a second semantics having the highest correlation with the first interpretation from at least one second semantics; and determine the second semantics as the target semantics.

[0263] Optionally, the calculation unit is further used to calculate a confidence score for the second answer to obtain a second score corresponding to the second answer;

[0264] The determination unit is further configured to determine the second reply as a final reply to the first question if the second score is not less than the first threshold value.

[0265] An embodiment of the present invention provides a question-and-answer device for sensitive content, the device comprising: an acquisition unit, used to acquire a first question; using a semantic tree to perform semantic extraction on the first question to obtain a first semantic corresponding to the first question; the first question contains sensitive content; a generation unit, used to use a prompt generator to generate a first prompt according to the first question and the first semantic; the first prompt is used to prompt a security answer to the first question; an input unit, used to input the first prompt into a preset large language model, and use the preset large language model to generate a first reply to the first question based on the first prompt; the preset large language model is trained according to the correspondence between sample sensitive instructions and security answers; using the above implementation scheme, the present invention first performs semantic extraction on the question input by the user to obtain the user's true intention, and then uses the prompt generator to generate a prompt for prompting the model to speak carefully, and finally uses the model to answer the question according to the prompt and the semantics of the question. This method can maximize the improvement of the model's security response to sensitive questions and can improve the accuracy of sensitive content detection.

[0266] Fig.14 A schematic diagram of the structure of a sensitive content question-answering device provided in an embodiment of the present invention. In practical applications, based on the same disclosed concept of the above embodiments, Fig.14 As shown, the sensitive content question-answering device 14 of this embodiment includes: a processor 1400 , a memory 1401 and a communication bus 1402 .

[0267] In the process of a specific embodiment, the acquisition unit 1300, the generation unit 1301, the input unit 1302, the screening unit, the data processing unit, the model training unit, the judgment unit, the determination unit and the calculation unit can be implemented by a processor 1400 located on the question-and-answer device 14 for sensitive content, and the processor 1400 can be at least one of an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing image processing device (DSPD), a programmable logic image processing device (PLD), a field programmable gate array (FPGA), a CPU, a controller, a microcontroller, and a microprocessor. It can be understood that for question-and-answer devices 14 for different sensitive contents, the electronic device used to implement the above-mentioned processor function can also be other, and this embodiment does not specifically limit it.

[0268] In an embodiment of the present invention, the communication bus 1402 is used to realize the connection and communication between the processor 1400 and the memory 1401; when the processor 1400 executes the running program stored in the memory 1401, the following sensitive content question and answer method is implemented:

[0269] Obtaining a first question; performing semantic extraction on the first question using a semantic tree to obtain a first semantic meaning corresponding to the first question; the first question contains sensitive content;

[0270] Using a prompt generator to generate a first prompt according to the first question and the first semantics; the first prompt is used to prompt a security answer to the first question;

[0271] The first prompt is input into a preset large language model, and a first response to the first question is generated based on the first prompt using the preset large language model; the preset large language model is trained based on the correspondence between the sample sensitive instructions and the security answer. Optionally, the processor 1400 is also used to obtain a sample sensitive instruction set; the sample sensitive instruction set is generated by multiple instructions related to sensitive content; a preset question-and-answer model is used to generate multiple corresponding answers for each sensitive instruction in the sample sensitive instruction set; at least one security answer is selected from the multiple answers, and a correspondence between the sample sensitive instruction and the security answer is generated based on the sample sensitive instruction set and at least one security answer corresponding to each sensitive instruction.

[0272] Optionally, processor 1400 is also used to obtain a sensitive content data set; the sample sensitive content data set includes multiple sensitive content data; using a preset instruction generation model, for each sensitive content data, at least one instruction required to obtain each sensitive content data is generated; based on multiple instructions corresponding to multiple sensitive content data, a sample sensitive instruction set is generated.

[0273] Optionally, processor 1400 is further used to obtain the correspondence between sample instructions and answers; reverse the correspondence between sample instructions and answers to obtain the correspondence between sample answers and instructions; reverse tune the initial instruction generation model according to the correspondence between sample answers and instructions to obtain a preset instruction generation model.

[0274] Optionally, processor 1400 is further used to perform security judgment on each of the multiple instructions using a preset judgment model to obtain multiple first judgment results corresponding to the multiple instructions; determine the instructions among the multiple instructions whose first judgment results are unsafe as sensitive instructions to obtain at least one sensitive instruction; and generate a sample sensitive instruction set based on the at least one sensitive instruction.

[0275] Optionally, processor 1400 is further used to perform security judgment on each instruction and the corresponding multiple answers using a preset judgment model, and obtain multiple second judgment results corresponding to the multiple answers; determine the answer whose second judgment result is safe among the multiple answers as a safe answer, and obtain at least one safe answer.

[0276] Optionally, processor 1400 is further used to calculate a confidence score for the first answer to obtain a first score corresponding to the first answer; when the first score is less than a first threshold value, use a semantic tree to determine at least one second semantic included in the first semantic; the second semantic is a subordinate semantic of the first semantic; use a prompt generator to generate a third prompt according to the first question and the target semantic in at least one second semantic; input the third prompt into a preset large language model, and use the preset large language model to generate a second reply to the first question based on the third prompt.

[0277] Optionally, the processor 1400 is further configured to use the prompt generator to determine a second semantics having the highest correlation with the first interpretation from at least one second semantics; and determine the second semantics as the target semantics.

[0278] Optionally, processor 1400 is further configured to calculate a confidence score for the second answer to obtain a second score corresponding to the second answer; when the second score is not less than a first threshold value, determine the second reply as the final reply to the first question.

[0279] An embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon. The computer-readable storage medium stores one or more programs. The one or more programs can be executed by one or more processors and are applied to a question-and-answer device for sensitive content. The computer program implements the question-and-answer method for sensitive content as described above.

[0280] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program code.

[0281] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0282] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0283] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0284] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A sensitive content question-answering method, characterized in that: The method comprises: Obtaining a first question; performing semantic extraction on the first question using a semantic tree to obtain a first semantics corresponding to the first question; the first question contains sensitive content; Using a prompt generator to generate a first prompt according to the first question and the first semantics; the first prompt is used to prompt a security answer to the first question; The first prompt is input into a preset large language model, and a first response to the first question is generated based on the first prompt using the preset large language model; the preset large language model is trained according to the correspondence between sample sensitive instructions and safe answers.

2. The sensitive content question-answering method according to claim 1, characterized in that: The method further comprises: Obtaining a sample sensitive instruction set; the sample sensitive instruction set is generated by a plurality of instructions related to sensitive content; Generate multiple corresponding answers for each sensitive instruction in the sample sensitive instruction set using a preset question-answering model; At least one security answer is selected from the multiple answers, and a correspondence between the sample sensitive instructions and the security answers is generated based on the sample sensitive instruction set and the at least one security answer corresponding to each sensitive instruction.

3. The sensitive content question-answering method according to claim 2, characterized in that: The method further comprises: Acquire a sensitive content data set; the sample sensitive content data set includes a plurality of sensitive content data; Using a preset instruction generation model, for each piece of sensitive content data, at least one instruction required to obtain the sensitive content data is generated; The sample sensitive instruction set is generated based on the multiple instructions corresponding to the multiple sensitive content data.

4. The sensitive content question-answering method according to claim 3, characterized in that: The method further comprises: Obtain the correspondence between sample instructions and answers; Reversing the correspondence between the sample instruction and the answer to obtain a correspondence between the sample answer and the instruction; The initial instruction generation model is reversely tuned according to the correspondence between the sample answers and the instructions to obtain the preset instruction generation model.

5. The sensitive content question-answering method according to claim 3 or 4, characterized in that: The generating the sample sensitive instruction set based on the multiple instructions corresponding to the multiple sensitive content data includes: Using a preset determination model to perform security determination on each of the plurality of instructions, to obtain a plurality of first determination results corresponding to the plurality of instructions; Determine the instructions whose first determination result is that they are unsafe among the plurality of instructions as sensitive instructions, and obtain at least one sensitive instruction; Based on the at least one sensitive instruction, the sample sensitive instruction set is generated.

6. The sensitive content question-answering method according to claim 2, characterized in that: The step of selecting at least one safe answer from the plurality of answers comprises: Using a preset determination model to perform security determination on each instruction and the corresponding multiple answers, to obtain multiple second determination results corresponding to the multiple answers; The answer whose second determination result is safe among the multiple answers is determined as a safe answer, to obtain the at least one safe answer.

7. The sensitive content question-answering method according to claim 1, characterized in that: The first reply includes a first answer to the first question. After generating the first reply to the first question based on the first prompt using the preset large language model, the method further includes: Calculating a confidence score for the first answer to obtain a first score corresponding to the first answer; In the case where the first score is less than a first threshold value, determining, by using the semantic tree, at least one second semantic included in the first semantic; the second semantic is a subordinate semantic of the first semantic; generating a third prompt using the prompt generator according to the first question and the target semantics in the at least one second semantics; The third prompt is input into a preset large language model, and a second response to the first question is generated based on the third prompt using the preset large language model.

8. The sensitive content question-answering method according to claim 7, characterized in that: The first reply also includes a first explanation for the first answer; and the method further includes: The prompt generator is used to determine a second semantics having the highest correlation with the first interpretation from the at least one second semantics; and the second semantics is determined as the target semantics.

9. The sensitive content question-answering method according to claim 7, characterized in that: The second reply includes a second answer to the first question and a second explanation to the second answer; the method further includes: Calculating a confidence score for the second answer to obtain a second score corresponding to the second answer; When the second score is not less than the first threshold value, the second reply is determined as a final reply to the first question.

10. A sensitive content question-answering device, characterized in that: The device comprises: An acquisition unit, configured to acquire a first question; perform semantic extraction on the first question using a semantic tree to obtain a first semantics corresponding to the first question; the first question contains sensitive content; A generating unit, configured to generate a first prompt according to the first question and the first semantics using a prompt generator; the first prompt is used to prompt a security answer to the first question; An input unit is used to input the first prompt into a preset large language model, and use the preset large language model to generate a first response to the first question based on the first prompt; the preset large language model is trained according to the correspondence between sample sensitive instructions and safe answers.

11. A sensitive content question-answering device, characterized in that: The device comprises: a processor, a memory and a communication bus; when the processor executes the running program stored in the memory, the method according to any one of claims 1 to 9 is implemented.

12. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images