A trusted question-and-answer method, system, electronic device, medium

Abstract

The application discloses a kind of trusted question and answer method, system, electronic equipment, medium, comprising: based on local document library construction vector database;Obtain inquiry content and inquiry document range;Inquiry content is input into content security detection model and is detected, when identifying that inquiry content contains violation, then output warning signal, reacquire inquiry content;Inquiry content is vectorized, and user inquiry vector is obtained;According to user inquiry vector, inquiry document range, vector database is searched, and query result text is obtained;Inquiry content and query result text are input into standard prompt template, then input into natural language big model, and return answer is obtained;Return answer is input into content security detection model and is detected, when identifying that return answer contains violation, then inquiry content, query result text, return answer and classification result are input into abnormal processing prompt template, then input into natural language big model, and return answer is obtained.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of AI response systems, and particularly relates to a trusted question and answer method and system, an electronic device and a medium. BACKGROUND

[0002] A domain question and answer system refers to a system for asking and answering questions in a certain professional domain. Compared with an open question and answer system, it has the characteristics of low implementation difficulty and high practical value because it focuses on a certain professional domain. In existing domain question and answer systems, due to the diversity and complexity of questions, traditional retrieval and matching methods often cannot meet the needs of users. The newly emerging method of fine-tuning a large model in a specific domain has the possibility of generating irrelevant or rule-violating content because it is essentially a probabilistic model. SUMMARY

[0003] Therefore, the application provides a trusted question and answer method, system, electronic device and medium, which detects the input / output content of a large model for compliance, avoids outputting illegal, rule-violating, unhealthy or unethical content, and realizes the trustworthiness and controllability of the question and answer system.

[0004] In a first aspect, the application provides a trusted question and answer method, which comprises:

[0005] Constructing a vector database based on a local document library;

[0006] Obtaining inquiry content and a range of inquiry documents; inputting the inquiry content into a pre-trained content safety detection model for detection; when the content safety detection model identifies that the inquiry content contains rule-violating content or instruction attack content, the content safety detection model outputs a warning signal, and the inquiry content is reacquired; when the inquiry content passes the detection of the content safety detection model, the inquiry content is vectorized to obtain a user inquiry vector;

[0007] Retrieving the vector database according to the user inquiry vector and the range of inquiry documents to obtain a query result text;

[0008] Inputting the inquiry content and the query result text into a standard prompt template and then into a natural language large model to obtain a returned answer; inputting the returned answer output by the natural language large model into the pre-trained content safety detection model for detection; when the content safety detection model identifies that the returned answer contains rule-violating content or instruction attack content, inputting the inquiry content, the query result text, the returned answer and the classification result output by the content safety detection model into an abnormality processing prompt template and then into the natural language large model to obtain a returned answer; when the returned answer passes the detection of the content safety detection model, outputting the final returned answer.

[0009] Further, the vector database is constructed based on the local document library, which includes:

[0010] a word number threshold, each document in the local document library is divided according to chapters and / or paragraphs, and the word number of each sub-document after division is less than the word number threshold;

[0011] each sub-document after division is named;

[0012] the text content of each sub-document after division is vectorized to obtain a text vector, and the text content, the text vector and the document name of each sub-document after division are stored in the vector database.

[0013] Further, the content security detection model includes:

[0014] The content security detection model adopts a BERT model structure and is trained by using a Chinese security evaluation public data set, and the trained content security detection model can identify and classify illegal content, instruction attack content and normal content.

[0015] Further, the vector database is searched according to the user inquiry vector and the inquiry document range to obtain a query result text, which includes:

[0016] The user inquiry vector is input as a query vector, and the inquiry document range is input as a vector search filtering condition into the vector database for searching, and the sub-document corresponding to the text vector with the highest similarity to the query vector in the vector database is taken as the query result text.

[0017] Further, the standard prompt template includes a question prompt word, a reference content input prompt word, a query result text, an inquiry content input prompt word and inquiry content.

[0018] Further, the abnormal processing prompt template includes a question prompt word, a reference content input prompt word, a query result text, an inquiry content input prompt word, inquiry content, a returned answer prompt word, a returned answer, a violation reason prompt word and a classification result output by the content security detection model.

[0019] Further, the trusted question and answer method further includes setting a total number of detection times, and when the returned answer fails to pass the content security detection model detection after reaching a specified detection number of times, a warning signal is generated, and the inquiry content is reacquired.

[0020] In a second aspect, an embodiment of the present application provides a trusted question and answer system, which includes:

[0021] A vector database construction module is configured to construct a vector database based on a local document library.

[0022] The pre-position safety detection module is used for acquiring inquiry content and a range of inquiry documents; the inquiry content is input into a pre-trained content safety detection model for detection; when the content safety detection model identifies that the inquiry content contains illegal content or instruction attack content, the content safety detection model outputs a warning signal, and the inquiry content is re-acquired; when the inquiry content passes the detection of the content safety detection model, the inquiry content is vectorized to obtain a user inquiry vector;

[0023] The query result text acquisition module is used for searching a vector database according to the user inquiry vector and the range of inquiry documents to obtain a query result text.

[0024] The natural language large model module is used for inputting the inquiry content and the query result text into a standard prompt template, inputting a generated prompt into a natural language large model, and obtaining a returned answer; or when the content safety detection model in the post-position safety detection module identifies that the returned answer contains illegal content or instruction attack content, the inquiry content, the query result text, the returned answer and a classification result output by the content safety detection model are input into an abnormality processing prompt template, a generated prompt is input into the natural language large model, and a returned answer is obtained.

[0025] The post-position safety detection module is used for inputting the returned answer output by the natural language large model into a pre-trained content safety detection model for detection; when the returned answer passes the detection of the content safety detection model, an ultimate returned answer is output.

[0026] In a third aspect, an electronic device is provided, including a memory and a processor, the memory is coupled with the processor; wherein the memory is used for storing program data, and the processor is used for executing the program data to realize the trusted question and answer method.

[0027] In a fourth aspect, a computer readable storage medium is provided, and a computer program is stored on the computer readable storage medium, and the program is executed by a processor to realize the trusted question and answer method.

[0028] Compared with the prior art, the present application has the following advantages: the present application provides a trusted question and answer method; the method of the present application avoids the generation of irrelevant or false information by the natural language large model by constraining the natural language large model with local documents in the form of prompts based on the natural language large model; the method of the present application simultaneously avoids the output of illegal, illegal, unhealthy or immoral content by constructing a content safety detection model to detect the input / output content of the natural language large model; and the trusted and controllable question and answer is realized through the above two means. BRIEF DESCRIPTION OF DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments description. Obviously, the drawings in the following description only some of the embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0030] Figure 1 A flowchart of a trusted question and answer method provided by the embodiments of the present application is shown.

[0031] Figure 2 A flowchart of content security detection on inquiry content provided by the embodiments of the present application is shown.

[0032] Figure 3 A flowchart of content security detection on returned answers provided by the embodiments of the present application is shown.

[0033] Figure 4 A schematic diagram of a trusted question and answer system provided by the embodiments of the present application is shown.

[0034] Figure 5 A schematic diagram of an electronic device provided by the embodiments of the present application is shown. DETAILED DESCRIPTION

[0035] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings of the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0036] It should be noted that the features in the following embodiments and implementation manners can be combined with each other without conflict.

[0037] As shown in the first aspect, the embodiments of the present application provide a trusted question and answer method based on a natural language large model and a local document library, which comprises the following steps: Figure 1

[0038] Step S1, constructing a vector database based on the local document library;

[0039] Specifically, the step S1 specifically comprises the following steps:

[0040] Step S101, setting a word threshold, and cutting each document in the local document library according to chapters and / or paragraphs, and the word number of each sub-document after cutting is less than the word threshold.

[0041] Step S102, naming each sub-document after cutting.​

[0042] Step S103, vectorize the text content of each sub-document after segmentation to obtain a text vector. Store the text content, text vector and document name corresponding to each sub-document after segmentation in the vector database.

[0043] In this example, the embedding model is used to vectorize the text content of each sub-document after segmentation to obtain a text vector.

[0044] Step S2, as shown in Figure 2 obtain the query content and the scope of the query document; input the query content into the pre-trained content security detection model for detection; when the content security detection model identifies that the query content contains illegal or instruction attack content, the content security detection model outputs a warning signal, and the query content is reacquired. When the query content passes the content security detection model detection, the query content is vectorized to obtain a user query vector.

[0045] Further, the scope of the query document is a list of query document names, and the scope of the query document is a full set or a subset of all sub-documents.

[0046] Further, the content security detection model adopts a bert model structure and is trained using a Chinese security evaluation public data set. The trained content security detection model can identify and classify illegal content, instruction attack content and normal content.

[0047] Step S3, search the vector database according to the user query vector and the scope of the query document to obtain a query result text.

[0048] Specifically, the user query vector is input into the vector database as a query vector and the scope of the query document is input into the vector database as a vector search filtering condition for searching. The sub-document corresponding to the text vector with the highest similarity to the query vector in the vector database is taken as the query result text.

[0049] Step S5, set a standard prompt template, input the query content and the query result text obtained in step S4 into the standard prompt template, and then input into the natural language large model to obtain a returned answer; input the returned answer output by the natural language large model into the pre-trained content security detection model for detection; when the content security detection model identifies that the returned answer contains illegal or instruction attack content, input the query content, the query result text, the returned answer and the classification result output by the content security detection model into an abnormal processing prompt template, and then input into the natural language large model to obtain a returned answer.

[0050] As shown in Figure 3As shown, the step S5 further includes: setting a total number of detection times, when the returned answer reaches a specified detection number of times and still fails to pass the content security detection model detection, a warning signal is generated, and the inquiry content is reacquired.

[0051] Further, the standard prompt template includes a question prompt word, a reference content input prompt word, a query result text, an inquiry content input prompt word, and inquiry content.

[0052] Further, the abnormal processing prompt template includes a question prompt word, a reference content input prompt word, a query result text, an inquiry content input prompt word, inquiry content, a returned answer prompt word, a returned answer, a violation reason prompt word, and a classification result output by the content security detection model.

[0053] Embodiment 1

[0054] In one embodiment, to build a classical literature knowledge base, four classic works in txt format are selected as the to-be-split-in-base documents, “multilingual-MiniLM-L12-v2--main” is selected as the embedding model, and faiss (Facebook AI Similarity Search) is selected as the vector storage query database.

[0055] First, the four classic works in txt format are split according to the intelligent document splitting method. The splitting principle is: the split sub-documents need to meet the following conditions: a single sub-document cannot exceed 5000 words, the same paragraph can only appear in one sub-document, and paragraph crossing is not allowed; a single sub-document can only contain one chapter. After splitting, record the original document name of each sub-document. The split sub-documents are vectorized using the embedding model, and then stored in the Elasticsearch vector database ES together with the sub-document content and the original document name.

[0056] In the embodiment, the content security data set collected includes multiple categories such as “dirty insults”, “bias discrimination”, “illegal crime”, “physical injury”, “mental health”, “property privacy”, “moral ethics”, “target hijacking”, “target hijacking”, “assign roles after instructions”, “unsafe instruction theme”, “ask with unsafe views”, and “negative induction”. Each category has a quantity of 5k-10k. In embodiment 1, a pre-trained bert-base-chinese model is selected as the base model. The bert-base-chinese model is used to construct a text multi-classification model, and then the content security data set is used to train the bert-base-chinese model to generate a content security detection model S.

[0057] In one embodiment, the user inquiry content Q1 is "Why does Lin Daiyu like Jia Baoyu?" and the designated inquiry range is "Dream of Red Mansions".

[0058] The inquiry content Q1 is input into the pre-trained content safety detection model for detection. If the content safety detection model detects that the inquiry content contains "dirty words and insults", the content safety detection model outputs a warning signal, and the inquiry content is reacquired. Specifically, the user is prompted with a warning message "Your recent inquiry may contain 'dirty words and insults' information. Please reorganize your language and ask the question again."

[0059] The user modifies his inquiry content Q2 to "What poem did Lin Daiyu write at the Crab Feast?", and sends the inquiry content Q2 into the content safety detection model for detection. The content safety detection model prompts that no abnormalities are detected, and the inquiry content Q2 is vectorized to generate the inquiry vector E1.

[0060] Then the inquiry vector E1 and the inquiry range "Dream of Red Mansions" are sent into the vector database ES as the query vector and the vector search filtering condition respectively. The vector database ES returns the retrieved content C1 of "Lin Xiaoxiang's Poem Contest for Chrysanthemum Poems, Shexue Hengwu's Satire, and Crab Ode" in the thirty-eighth chapter of "Dream of Red Mansions". The inquiry content Q2 and the retrieved text content C1 are sent into the standard prompt template to generate a prompt as follows:

[0061]

[0062]

[0063] The generated prompt is sent into the natural language large model chatgpt-3.5 to obtain the answer A1 as follows:

[0064]

[0065] The generated answer A1 is input into the pre-trained content safety detection model for detection. Specifically, the detection times N is set to 1, and then the answer is sent into the content safety detection model. After passing the content safety detection model, the final user is returned.

[0066] Embodiment 2

[0067] In another embodiment, the user inquiry content Q3 is "How did Li Kui deal with the daughter of Di Taigong in the chapter of Black Whirlwind Joe Catching Ghosts?" and the designated inquiry range is "Water Margin".

[0068] The inquiry content Q3 is input into the pre-trained content safety detection model for detection, and no abnormalities are detected. Then the inquiry content Q3 is vectorized to generate the inquiry vector E2.

[0069] Then the query vector E2 and the backup query range 《Water Margin》 are sent into the vector database ES as the query vector and the vector search filtering condition respectively. The vector database ES returns the retrieved content C2 of the 73rd chapter of Water Margin 《Black Whirlwind Jia Catching Ghosts in Liangshan Marshes》. The query content Q3 and the retrieved text content C2 are sent into the standard prompt template to generate a prompt as follows:

[0070]

[0071] The generated prompt is input into the natural language large model chatgpt-3.5 to obtain the answer A2.

[0072] The generated answer A2 is subjected to content safety detection. Specifically, first, the detection number N is set to N = 1, and then the generated answer A2 is input into the pre-trained content safety detection model. It is determined that there is suspected "physical injury" content in the content safety detection model. The detection number N is set to N = N + 1 for iterative detection. The reference content C2, the query content Q3, the model answer content A2, and the "physical injury" label are input into the abnormal processing prompt template to generate a new prompt as follows:

[0073]

[0074] The new prompt is input into the natural language large model to obtain the answer A3. The generated answer A3 is input into the content safety detection model, which passes the content safety detection model and returns to the final user.

[0075] As shown in Figure 4 , in a second aspect, the embodiments of the present application also provide a trusted question and answer system, which comprises:

[0076] A vector database construction module is configured to construct a vector database based on a local document library.

[0077] A pre-safety detection module is configured to obtain query content and backup query document range. The query content is input into a pre-trained content safety detection model for detection. When the content safety detection model identifies that the query content contains illegal content or instruction attack content, the content safety detection model outputs a warning signal, and the query content is reacquired. When the query content passes the content safety detection model detection, the query content is vectorized to obtain a user query vector.

[0078] The natural language large model module is configured to input the inquiry content and the query result text into a standard prompt template, input a generated prompt into a natural language large model, and obtain a returned answer; or when the content safety detection model in the post-security detection module identifies that the returned answer contains illegal content or instruction attack content, input the inquiry content, the query result text, the returned answer, and a classification result output by the content safety detection model into an abnormal processing prompt template, input a generated prompt into the natural language large model, and obtain a returned answer.

[0079] The post-security detection module is configured to input the returned answer output by the natural language large model into a pre-trained content safety detection model for detection, and output a final returned answer when the returned answer passes the detection of the content safety detection model.

[0080] As shown in Figure 5 The electronic device provided by the embodiments of the present application includes a memory 101 configured to store one or more programs, and a processor 102. When the one or more programs are executed by the processor 102, the method of any one of the above first aspects is implemented.

[0081] The electronic device further includes a communication interface 103. The memory 101, the processor 102, and the communication interface 103 are electrically connected to each other directly or indirectly to realize the transmission or interaction of data. For example, these elements can be electrically connected to each other through one or more communication buses or signal lines. The memory 101 can be used to store software programs and modules. The processor 102 performs various functional applications and data processing by executing the software programs and modules stored in the memory 101. The communication interface 103 can be used for signaling or data communication with other node devices.

[0082] The memory 101 can be, but is not limited to, a random access memory 101 (RAM), a read-only memory 101 (ROM), a programmable read-only memory 101 (PROM), an erasable programmable read-only memory 101 (EPROM), an electrically erasable programmable read-only memory 101 (EEPROM), and the like.

[0083] The processor 102 can be an integrated circuit chip having signal processing capability. The processor 102 can be a general-purpose processor 102, including a central processing unit 102 (CPU), a network processor 102 (NP), etc.; and can also be a digital signal processor 102 (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0084] The processor 102 can be an integrated circuit chip having signal processing capability. The processor 102 can be a general-purpose processor 102, including a central processing unit 102 (CPU), a network processor 102 (NP), etc.; and can also be a digital signal processor 102 (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0085] In the embodiments of the present application, it should be understood that the disclosed method and system can also be implemented by other means. The above-described method and system embodiments are only illustrative, for example, the flowchart and block diagram in the drawings show the possible implementation architecture, function and operation of the method and system, method and computer program product according to the embodiments of the present application. In this regard, each block in the flowchart or block diagram can represent a module, a program segment or a part of code, which contains one or more executable instructions for implementing the specified logic function. It should also be noted that in some alternative implementations, the functions noted in the block can occur in different order than that shown in the figure. For example, two consecutive blocks can actually be executed substantially in parallel, and sometimes they can be executed in reverse order, depending on the function involved. It should also be noted that each block in the block diagram and / or flowchart, and the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for performing the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0086] In addition, each functional module in the various embodiments of the present application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0087] In another aspect, the embodiments of the present application provide a computer readable storage medium, which stores a computer program. The computer program is executed by the processor 102 to implement the method of any one of the above first aspect. The functions, if implemented in the form of software function modules and sold or used as independent products, can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application, essentially or in part, or parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for making a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in the embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk, and various other media that can store program codes.

[0088] The above embodiments are only used to illustrate the design ideas and characteristics of the present application, and the purpose is to enable those skilled in the art to understand the present application and to implement it, and the protection scope of the present application is not limited to the above embodiments. Therefore, any equivalent changes or modifications made according to the principles disclosed by the present application are within the scope of protection of the present application.

Claims

1. A trusted challenge-response method, characterized by The method comprises: constructing a vector database based on a local document library; obtaining inquiry content and a range of inquiry documents; inputting the inquiry content into a pre-trained content security detection model for detection; when the content security detection model identifies that the inquiry content contains illegal content or instruction attack content, the content security detection model outputs a warning signal, and the inquiry content is reacquired; when the inquiry content passes the detection of the content security detection model, the inquiry content is vectorized to obtain a user inquiry vector; retrieving the vector database according to the user inquiry vector and the range of inquiry documents to obtain query result text; inputting the inquiry content and the query result text into a standard prompt template, inputting the generated prompt into a natural language large model to obtain a returned answer; inputting the returned answer output by the natural language large model into a pre-trained content security detection model for detection; when the content security detection model identifies that the returned answer contains illegal content or instruction attack content, inputting the inquiry content, the query result text, the returned answer and the classification result output by the content security detection model into an abnormal processing prompt template, inputting the generated prompt into the natural language large model to obtain a returned answer; when the returned answer passes the detection of the content security detection model, outputting the final returned answer; The abnormal processing prompt template comprises: a question prompt word, a reference content input prompt word, query result text, an inquiry content input prompt word, inquiry content, a returned answer prompt word, a returned answer, an illegal reason prompt word and a classification result output by the content security detection model.

2. The trusted question-and-answer method of claim 1, wherein, Constructing a vector database based on a local document library comprises: setting a word number threshold, dividing each document in the local document library according to chapters and / or paragraphs, and the word number of each sub-document after division is less than the word number threshold; naming each sub-document after division; vectorizing the text content of each sub-document after division to obtain a text vector, and storing the text content, the text vector and the document name of each sub-document after division in the vector database.

3. The trusted question answering method of claim 1, wherein, The content security detection model comprises: The content security detection model adopts a bert model structure and is trained using a Chinese security evaluation public data set, and the trained content security detection model can identify and classify illegal content, instruction attack content and normal content.

4. The trusted question answering method of claim 1, wherein, Retrieving the vector database according to the user inquiry vector and the range of inquiry documents to obtain query result text comprises: inputting the user inquiry vector as a query vector and the range of inquiry documents as a vector search filtering condition into the vector database for retrieval, and taking the sub-document corresponding to the text vector with the highest similarity to the query vector in the vector database as the query result text.

5. The trusted question answering method of claim 1, wherein, The standard prompt template comprises: a question prompt word, a reference content input prompt word, query result text, an inquiry content input prompt word and inquiry content.

6. The trusted question answering method of claim 1, wherein, The trusted question and answer method further comprises: setting a total number of detection times, generating a warning signal and reacquiring inquiry content when the returned answer fails to pass the detection of the content security detection model after reaching a specified number of detection times.

7. A trusted challenge-response system, characterized in that The system comprises: A vector database construction module is configured to construct a vector database based on a local document library; The pre-security detection module is configured to acquire the inquiry content and the scope of the inquiry document; input the inquiry content into a pre-trained content security detection model for detection; when the content security detection model identifies that the inquiry content contains illegal content or instruction attack content, the content security detection model outputs a warning signal, and the inquiry content is reacquired; when the inquiry content passes the detection of the content security detection model, the inquiry content is vectorized to obtain a user inquiry vector; The query result text acquisition module is configured to search the vector database according to the user inquiry vector and the scope of the inquiry document to obtain a query result text; The natural language large model module is configured to input the inquiry content and the query result text into a standard prompt template, input the generated prompt into a natural language large model, and obtain a returned answer; when the content security detection model in the post-security detection module identifies that the returned answer contains illegal content or instruction attack content, the inquiry content, the query result text, the returned answer, and the classification result output by the content security detection model are input into an abnormal processing prompt template, the generated prompt is input into the natural language large model, and the returned answer is obtained; wherein the abnormal processing prompt template includes a question prompt word, a reference content input prompt word, a query result text, an inquiry content input prompt word, inquiry content, a returned answer prompt word, a returned answer, an illegal reason prompt word, and a classification result output by the content security detection model; The post-security detection module is configured to input the returned answer output by the natural language large model into a pre-trained content security detection model for detection, and output a final returned answer when the returned answer passes the detection of the content security detection model.

8. An electronic device comprising a memory and a processor, characterized in that, The memory is coupled with the processor; wherein the memory is configured to store program data, and the processor is configured to execute the program data to implement the trusted question and answer method of any one of claims 1-7.

9. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the trusted question and answer method of any one of claims 1-7.

Images