A generative large model document library question answering method based on reasoning prompts

Abstract

The application discloses a generative large model document library question and answer method based on reasoning prompts, and relates to the field of natural language processing analysis, and comprises the following steps: firstly, data processing and article vector retrieval library construction are performed based on a generative large model; secondly, information enhancement is performed on user question content; thirdly, based on the article vector retrieval library, a candidate article set is obtained by performing vector retrieval on the enhanced user question; fourthly, corresponding prompt libraries are constructed for different types of questions; fifthly, the user question is classified based on the recognition ability of the generative large model on the text type; sixthly, model input prompts of the user question are obtained based on the prompt libraries and the candidate article set; and finally, the model input prompts are input into the generative large model to obtain answers to the user question.

Description

Technical Field

[0001] This invention relates to the field of natural language processing analysis, specifically to a generative large model document library question answering method based on reasoning prompts. Background Technology

[0002] The statements in this section are provided only as background information in connection with this disclosure and may not constitute prior art.

[0003] In natural language processing (NLP) knowledge acquisition analysis, there is a large amount of unstructured document data, characterized by diverse structures, high dimensionality, large data volume, complex semantic structures, and strong coupling relationships between paragraphs. Question answering, as an important method of information acquisition, relies heavily on unstructured document data, currently primarily applied in information retrieval and multi-document knowledge acquisition. However, the sheer volume and complex semantic structure of document data make accurately understanding user question intent, quickly retrieving candidate article sets, and precisely locating answers within articles increasingly complex and challenging. Therefore, establishing an efficient and accurate intelligent question answering system based on unstructured document data to achieve key information acquisition from large-scale document data is of significant research value.

[0004] Currently, document-based question answering primarily employs extractive question answering and information retrieval methods based on pre-trained models. Extractive question answering methods extract document features using pre-trained models, generating vector representations of states and actions. Activation functions are then used to predict the start and end positions of the answer, thus retrieving it. However, because it extracts answers from article content, it has significant limitations in handling inference-based questions and also presents challenges in answering questions across paragraphs. In practical applications, users often seek information from multiple articles, and inference-based questions are common. Traditional methods can only extract answers from existing articles and cannot infer from user questions and known content. Therefore, a document-based question answering method that enables summarization and inference should be developed, driven by actual user needs. Summary of the Invention

[0005] The purpose of this invention is to address the problems existing in the prior art by providing a generative large-scale document library question answering method based on inference prompts. This method quickly analyzes the user's question type using an intent understanding model, enhances key question information through data augmentation, and recalls candidate documents related to the user's question based on vector retrieval. Furthermore, to improve the accuracy of the model's returned answers, different inference prompts are constructed according to the question type. The prompt, constructed using the user's question, inference prompts, and the retrieved candidate document set, serves as the input to the generative large-scale model. This enables the accurate acquisition of answers to factual and inference-based questions based on a document library, aiming to solve the problem of acquiring answers to factual and inference-based questions in both single and cross-document formats.

[0006] The technical solution of the present invention is as follows:

[0007] A generative large model document library question answering method based on reasoning prompts includes:

[0008] Step S1: Data processing and article vector retrieval library construction based on generative large model;

[0009] Step S2: Enhance the information in the user's question;

[0010] Step S3: Based on the article vector retrieval library, obtain a set of candidate articles by performing vector retrieval on the enhanced user question;

[0011] Step S4: Build corresponding suggestion libraries for different types of problems;

[0012] Step S5: Based on the generative large model's ability to identify text types, classify the user questions;

[0013] Step S6: Based on the prompt library and candidate article set, obtain the model input prompt for the user question;

[0014] Step S7: Input the model into prompt, then input it into the generative large model to obtain the answer to the user's question.

[0015] Further, step S1 includes:

[0016] Based on the segmentation and sentence segmentation, the document library data is vectorized using the BGE Chinese word embedding model to construct an article vector retrieval library.

[0017] Furthermore, the vectorization process includes:

[0018] Step S11: Segment the unstructured documents in the document library by paragraph, and then segment each paragraph by sentence;

[0019] Step S12: Based on the BGE Chinese word embedding model, the segmented sentences are vectorized to obtain the sentence vectors of the corresponding articles;

[0020] Step S13: Store the vectorized sentence vectors, sentences, and paragraphs to complete the construction of the article vector retrieval library.

[0021] Further, step S2 includes:

[0022] Information enhancement is performed on user question content based on keyword extraction, keyword completion, or stop word methods.

[0023] Further, step S3 includes:

[0024] Based on cosine similarity, the similarity between the user question and the sentence vector is measured; if the calculated similarity result is greater than a set threshold, a candidate knowledge set is returned.

[0025] Further, step S4 includes:

[0026] For reasoning-type questions, a reasoning hint library is built.

[0027] Further, step S5 includes:

[0028] Determine whether the user's question is a reasoning question.

[0029] Further, step S6 includes:

[0030] Step S61: If the user question is a reasoning question, then construct a model input prompt based on the reasoning prompt library, the candidate article set, and the user question;

[0031] Step S62: If the user question is a non-reasoning question, then construct the model input prompt based on the candidate article set and the user question.

[0032] Further, step S61 includes:

[0033] Based on the inference hint library, inference hints for the user's question are obtained through similarity matching. Based on the inference hints, the candidate article set, and the user's question, a model input prompt is constructed.

[0034] Furthermore, the construction of the reasoning hint library requires the acquisition of high-quality questions and hint templates;

[0035] The selection of high-quality problems is based on a screening method using uncertainty index calculation.

[0036] The beneficial effects of this invention are: firstly, it enables generative large models to answer questions based on external databases; secondly, it constructs a high-quality reasoning hint library based on uncertainty calculation methods and contrastive learning techniques; and thirdly, it improves the accuracy of large models in factual question answering through question candidate knowledge acquisition, question type identification, and high-quality reasoning hints. Attached Figure Description

[0037] Figure 1 This is a flowchart illustrating the overall process of a generative large model document library question-answering method based on reasoning prompts.

[0038] Figure 2 The process of building a high-quality reasoning hint library. Detailed Implementation

[0039] It should be noted that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0040] The features and performance of the present invention will be further described in detail below with reference to embodiments.

[0041] Example 1

[0042] Please see Figure 1 A generative large model document library question answering method based on reasoning prompts, specifically including the following steps:

[0043] Step S1: Data processing and article vector retrieval library construction based on generative large model;

[0044] Step S2: Enhance the information in the user's question;

[0045] Step S3: Based on the article vector retrieval library, obtain a set of candidate articles by performing vector retrieval on the enhanced user question;

[0046] Step S4: Build corresponding suggestion libraries for different types of problems;

[0047] Step S5: Based on the generative large model's ability to identify text types, classify the user questions;

[0048] Step S6: Based on the prompt library and candidate article set, obtain the model input prompt for the user question;

[0049] Step S7: Input the model into prompt, then input it into the generative large model to obtain the answer to the user's question.

[0050] In this embodiment, it should be noted that the generative big model adopts a general generative big model, which has strong natural language understanding ability, but lacks interpretability in factual question answering and has the phenomenon of generating incorrect answers. Therefore, this embodiment uses an external knowledge base to utilize the general understanding ability of the generative big model so that the model can generate answers to questions based on the given article.

[0051] In document-based question answering, the first step is to recall a set of candidate articles related to the user's question. In this embodiment, candidate articles are obtained based on vector retrieval, so a relevant document vector library is constructed according to actual needs.

[0052] Specifically, step S1 includes:

[0053] Based on the segmentation and sentence segmentation, the document library data is vectorized using the BGE Chinese word embedding model to construct an article vector retrieval library.

[0054] Specifically, the vectorization process includes:

[0055] Step S11: Segment the unstructured documents in the document library by paragraph, and then segment each paragraph by sentence; that is, user questions generally correspond to one or more sentences in an article, and vector retrieval is more accurate in matching concise and condensed articles, so unstructured documents are segmented by paragraph, and then segment each paragraph by sentence.

[0056] Step S12: Based on the BGE Chinese word embedding model, the segmented sentences are vectorized to obtain the sentence vectors of the corresponding articles;

[0057] Step S13: Store the vectorized sentence vectors, sentences, and paragraphs to complete the construction of the article vector retrieval library.

[0058] In this embodiment, specifically, step S2 includes:

[0059] Information enhancement is achieved by using keyword extraction, keyword completion, or stop word removal methods to augment the content of user questions. Directly using user questions for vector retrieval to obtain candidate knowledge may result in irrelevant search results, primarily due to the high generalization of user questions and the presence of numerous stop words. Therefore, this embodiment enhances information through keyword extraction, keyword completion, and stop word removal. Furthermore, by associating extracted and completed keywords with user questions, vector retrieval of candidate knowledge can be performed, thereby improving search accuracy.

[0060] In this embodiment, specifically, step S3 includes:

[0061] Based on cosine similarity, the similarity between the user question and the sentence vector is measured; if the calculated similarity result is greater than a set threshold, a candidate knowledge set is returned.

[0062] In this embodiment, for details, please refer to... Figure 2 Step S4 includes:

[0063] For reasoning-type questions, a reasoning hint library is constructed. It should be noted that the construction of the reasoning hint library requires the acquisition of high-quality questions and hint templates. The selection of high-quality questions is based on a method of filtering using uncertainty index calculation.

[0064] Specifically, for reasoning-type questions, in order to improve the accuracy of their answers, a high-quality reasoning hint library is to be established. High-quality reasoning hints directly affect the performance of generative large models, so different input instructions are constructed according to different types of questions. The construction of a high-quality reasoning hint library requires the acquisition of high-quality questions and hint templates. Reducing the uncertainty of the model is beneficial to improving the performance of the model. Therefore, the selection of high-quality questions is based on the calculation of uncertainty index.

[0065] Suppose there are k difficult questions identified based on the initial screening and feedback from the historical question database, denoted by Q. If the large model is called m times, each question will yield m answers, denoted by A. The accuracy of the generated results is determined based on Rouge-L and manual verification. If Rouge-L is greater than a specified threshold... If the answer is correct, then the question is considered correct; otherwise, it is considered incorrect.

[0066] Set a question The uncertainty index is The uncertainty value of the corresponding answer is... Assuming each question calls the odd number of largest models, the number of possible answers, m, will be odd. The median of these answers will be the correct answer. The possible values ​​of , and incorrect answers Take the value of the current position index, then , The calculation formula can be expressed as:

[0067]

[0068]

[0069] in, For the first The standard answer to this question It is the first The question is number one. The answer is the second one. Let be the uncertainty value corresponding to the q-th problem. This represents the number of distinct answers to the current question, and the average of these values ​​represents the uncertainty value. , , The larger the value, the higher the uncertainty.

[0070] Based on uncertainty indicators, a high-quality candidate set of questions can be screened and obtained. These questions are then labeled to form reasoning hint templates. During the labeling process, the same question is labeled in different ways (question + answer parsing process). To obtain high-quality reasoning hint templates from the labels, testing is conducted based on test data. A prompt for the input of a large model is constructed using the method of (reasoning hints + test dataset). The accuracy of the test set under different reasoning hints for the same type of question is calculated. Reasoning hints with higher accuracy are selected and added to the reasoning hint library. Subsequently, the hint learning method can be used for reasoning questions to improve their question-answering accuracy.

[0071] In this embodiment, specifically, step S5 includes:

[0072] Determine whether the user question is a reasoning question; that is, different prompts need to be designed for different types of questions. Therefore, the user question type is first identified. Based on the type identification capability of the generative large model, a prompt template for the model input is constructed. Preferably, it can be designed as "Please determine whether the following question is a reasoning question. Please answer yes or no. Question: xxx". This method can complete the identification of the user question type.

[0073] In this embodiment, specifically, step S6 includes:

[0074] Step S61: If the user question is a reasoning question, then a model input prompt is constructed based on the reasoning hint library, the candidate article set, and the user question; specifically, step S61 includes:

[0075] Based on the inference hint library, inference hints for the user's question are obtained through similarity matching. Based on the inference hints, the candidate article set, and the user's question, a model input prompt is constructed.

[0076] Step S62: If the user question is a non-reasoning question, then construct the model input prompt based on the candidate article set and the user question;

[0077] Preferably, for reasoning-type questions, the model input prompt can be constructed as "Hint xxx. + Please answer the questions according to the above hint and the following article. + Article: xxx. Question: xxx?"; for other types of questions (non-reasoning-type questions), the model input prompt can be constructed as "Please answer the questions based on the following article. Article: xxx. Question: xxx".

[0078] Example 2

[0079] Example 2 is a specific application of a generative large model document library question answering method based on reasoning prompts proposed in Example 1.

[0080] 1. Data Preparation

[0081] See Table 1 for the document library data assumed in this example. This example uses BGE Chinese word embedding model to vectorize the document data in the form of segments and sentences to build an article vector retrieval library.

[0082] Meanwhile, suppose the user inputs the question: "How many weapons did Army W and Army H lose respectively in the May xxx conflict?"

[0083] Table 1. Document library data assumed in this implementation.

[0084]

[0085] 2. Enhanced problem information

[0086] Based on methods such as keyword extraction, keyword completion, and stop word removal, information enhancement was performed on user questions. Specifically, the time "May" was completed, resulting in "May 2023". Stop words were removed through word segmentation. The final user question used for candidate article retrieval was "What was the number of weapons lost by Army W and Army H in the xxx conflict in May 2023?".

[0087] Based on vector retrieval, the candidate article set can be obtained as {Article 1: "May 1st xxx conflict news", Article 2: "May 2nd xxx conflict news"}.

[0088] 3. Understanding the Intent of the Problem

[0089] Based on the generative large model's ability to identify text types, the user question is classified. The model input is: "Please determine whether the following question is a reasoning question. Please answer yes or no. Question: How many weapons did Army W and Army H lose respectively in the May xxx conflict?" The model returns "Yes". Therefore, the user input question can be determined to be a reasoning question.

[0090] 4. Construct reasoning prompts

[0091] Since this user's question is a reasoning question, the reasoning hints for this question are obtained through similarity matching based on the constructed reasoning hint library. Suppose the obtained reasoning hints are: "Question: How many people were injured or killed in the xxx conflict?; Answer: In the xxx conflict, the W army sent a total of 200 people, and 132 people returned safely. Therefore, the number of casualties is 200-13 equals 68 people."

[0092] Based on the obtained reasoning hints, the model input prompt for this problem can be designed as follows:

[0093] Reasoning hint: "Question: What was the number of casualties in the xxx conflict?; Answer: In the xxx conflict, the W army sent a total of 200 people, and 132 people returned safely. Therefore, the number of casualties is 200-13 equals 68 people."

[0094] Please answer the new question based on the above reasoning clues and the following article content.

[0095] Article content: "Article 1: 'News Flash, May 1, 2023: ...'; Article 2: 'News Flash, May 2, 2023: ...';

[0096] New question: "What were the numbers of weapons lost by Army W and Army H respectively in the May xxx conflict?"

[0097] 5. Model Output

[0098] The model input prompt constructed in the above manner outputs the following results: "In May 2023, Army W dispatched 12 fighter jets, 8 of which returned, resulting in a loss of 12-8 equals 4 aircraft; Army W dispatched 10 tanks, 5 of which returned, resulting in a loss of 10-5 equals 5 tanks; Army Y dispatched 8 fighter jets, 4 of which returned, resulting in a loss of 8-4 equals 4 aircraft; Army Y dispatched 20 tanks, 17 of which returned, resulting in a loss of 20-7 equals 3 tanks."

[0099] The embodiments described above merely illustrate specific implementation methods of this application, and while the descriptions are detailed and specific, they should not be construed as limiting the scope of protection of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the technical solution of this application, and these modifications and improvements all fall within the scope of protection of this application.

[0100] This background section is provided to generally present the context of the invention. The work of the currently named inventors, the work to the extent described in this background section, and aspects of this section that did not constitute prior art at the time of application are neither expressly nor impliedly acknowledged as prior art to the invention.

Claims

1. An inference prompt-based generative large model document library question and answer method, characterized in that, The method comprises the following steps: Step S1: data processing based on a generative large model and construction of an article vector retrieval library; Step S2: information enhancement on the user question content; Step S3: based on the article vector retrieval library, a candidate article set is obtained by vector retrieval on the enhanced user question; Step S4: for different types of questions, a corresponding prompt library is constructed; Step S5: based on the text type recognition ability of the generative large model, the user question is classified; Step S6: based on the prompt library and the candidate article set, a model input prompt for the user question is obtained; Step S7: the model input prompt is input into the generative large model to obtain an answer to the user question. The step S4 comprises: For reasoning questions, a reasoning prompt library is constructed. The step S5 comprises: It is judged whether the user question is a reasoning question. The step S6 comprises: Step S61: if the user question is a reasoning question, a model input prompt is constructed based on the reasoning prompt library, the candidate article set and the user question; Step S62: if the user question is a non-reasoning question, a model input prompt is constructed based on the candidate article set and the user question. The step S61 comprises: Based on the reasoning prompt library, the reasoning prompt for the user question is obtained by similar matching, and the model input prompt is constructed based on the reasoning prompt, the candidate article set and the user question. The construction of the reasoning prompt library needs to obtain high-quality questions and prompt templates. The selection of the high-quality questions is based on the screening by the uncertain index calculation method, which comprises: Assume that there are k difficult problems based on the preliminary screening and feedback in the historical problem library, denoted by Q Then call the large model m times to get m answers for each question, denoted by A The accurate determination of the generated result is based on Rouge-L and artificial confirmation determination method. If Rouge-L is greater than a specified threshold Then determine that the question is the correct answer, otherwise it is the wrong answer; The uncertainty index of the question is , the uncertainty value of the corresponding answer is , assuming that each question calls the large model an odd number of times, the number of question answers m is odd, and the median of the number of answers is the correct answer , the value of the of the wrong answer is , and the value of the current position sequence number is , The calculation formula is: in, For the first The standard answer to this question It is the first The question is number one. The answer is the second one. Let be the uncertainty value corresponding to the q-th problem. This represents the number of distinct answers to the current question, and the average of these values ​​represents the uncertainty value. , , The larger the value, the higher the uncertainty.

2. The method of claim 1, wherein the method further comprises: The step S1 comprises: The document library data is vectorized based on the BGE Chinese word embedding model in the form of segmentation and sentence segmentation to construct an article vector retrieval library.

3. The method of claim 2, wherein the method further comprises: The vectorization processing comprises: Step S11: the unstructured documents in the document library data are segmented by paragraphs, and the paragraphs are segmented by sentences; Step S12: the segmented sentences are vectorized based on the BGE Chinese word embedding model to obtain the sentence vectors of the corresponding articles; Step S13: the vectorized sentence vectors, sentences and paragraphs are stored to complete the construction of the article vector retrieval library.

4. The method of claim 1, wherein the method further comprises: The step S2 comprises: The user question content is information-enhanced based on the keyword extraction, keyword completion or stop word method.

5. The method of claim 1, wherein the method further comprises: The step S3 comprises: The similarity between the user question and the sentence vector is measured based on the cosine similarity; if the calculated similarity result is greater than a set threshold, the candidate knowledge set is returned.

Images