Question and answer method and related apparatus

Abstract

The application provides a question and answer method and related device, the question and answer method comprises: obtaining a user question input by a user on a front-end page; reasoning the user question to obtain a thinking chain of the user question, the thinking chain comprising a plurality of reasoning steps for solving the user question; searching for supporting text matching each reasoning step in the thinking chain in a text database; obtaining an answer to the user question based on the user question and the searched supporting text; and displaying the answer on the front-end page. In the application, the user question is reasoned to generate a thinking chain of the user question, and the supporting text is searched in the text database through the thinking chain. Since the semantics of the thinking chain is closer to the semantics of the supporting text than the semantics of the question, the difference in semantics can be reduced when matching, the accuracy of text searching is improved, and the accuracy of answering the question is further improved.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of artificial intelligence technology, and in particular to a question-answering method and related apparatus. Background Technology

[0002] A question-answering system is an artificial intelligence system that can understand user-inputted questions and extract relevant information from text databases to generate answers. Current question-answering methods involve a process of first retrieving, then thinking, and finally answering. That is, it first retrieves supporting text from a text database based on the question, then considers the question and supporting text to arrive at and output the answer. Existing text retrieval methods fall into two categories: keyword matching and semantic matching. Keyword matching is too simplistic and ineffective. Semantic matching, on the other hand, can also fail when the question is complex, as semantic differences may exist between the question and supporting text, leading to poor matching results. Summary of the Invention

[0003] This invention provides a question-answering method and related apparatus to address the problem of poor matching between retrieved supporting text and questions in existing question-answering methods.

[0004] To solve the above-mentioned technical problems, the present invention is implemented as follows:

[0005] In a first aspect, embodiments of the present invention provide a question-and-answer method, including:

[0006] Retrieve user questions entered by the user on the front-end page;

[0007] Reasoning is performed on the user problem to obtain the thought chain of the user problem, which includes multiple reasoning steps for solving the user problem;

[0008] Retrieve supporting text from the text database that matches each reasoning step in the thought chain;

[0009] Based on the user's question and the retrieved supporting text, the answer to the user's question is obtained;

[0010] The answer is displayed on the front-end page.

[0011] Optionally, reasoning is performed on the user's question to obtain the thought chain of the user's question, including:

[0012] Based on the user question and pre-set instructions, a first prompt is constructed, the instructions being used to guide the first inference model to generate multiple inference steps to solve the user question;

[0013] The first prompt is input into the first inference model to obtain the thought chain of the user's question output by the first inference model.

[0014] Optionally, reasoning is performed on the user's question to obtain the thought chain of the user's question, including:

[0015] Obtain examples related to the user question, including examples of user questions and thought chains of the user question examples;

[0016] Based on the user question, the example, and the pre-set instructions, a second prompt is constructed, the instructions being used to guide the first inference model to generate multiple inference steps to solve the user question;

[0017] The second prompt is input into the first reasoning model to obtain the thought chain of the user's question output by the first reasoning model.

[0018] Optionally, retrieve supporting text from a text database that matches each reasoning step in the thought process, including:

[0019] Based on the reasoning steps in the thought chain, a third prompt is constructed, which is used to guide the second reasoning model to retrieve supporting text from the text database that matches the reasoning steps of the thought chain;

[0020] The third prompt is input into the second reasoning model to obtain the supporting text that matches the reasoning steps of the thought chain, output by the second reasoning model.

[0021] Optionally, the method further includes:

[0022] Obtain a training set, which includes multiple sets of sample data. Each set of sample data includes: user question samples, reasoning steps in the thought chain samples of the user question samples, and text samples. The labels of the text samples are one of the following: supporting text corresponding to the reasoning steps, non-supporting text in the supporting text set of the user question samples, and irrelevant text.

[0023] The sample data is input into the inference model to be trained to obtain the inference result output by the inference model. The inference result is used to indicate whether the text sample in the sample data is supporting text that matches the inference step in the sample data. The loss is determined based on the inference result and the label of the text sample. The inference model is fine-tuned based on the loss to obtain the trained second inference model.

[0024] Optionally, the method further includes:

[0025] Acquiring sample data; the acquisition of sample data includes:

[0026] Obtain a thought chain sample and supporting text set for user question samples;

[0027] Based on each reasoning step in the thought chain sample and the supporting text in the supporting text set, a fourth prompt is constructed. The fourth prompt is used to guide the third reasoning model to identify whether the supporting text matches the reasoning steps of the thought chain sample.

[0028] The fourth prompt is input into the third reasoning model to obtain the matching results between the supporting text output by the third reasoning model and each reasoning step in the thought chain sample;

[0029] The tags for the supporting text are determined based on the matching results.

[0030] Optionally, the second reasoning model is one of the following: a reasoning model based on natural language understanding, a reasoning model based on natural language generation, or a reasoning model based on knowledge graphs.

[0031] Secondly, embodiments of the present invention provide a question-and-answer system, including:

[0032] The question retrieval module is used to retrieve user questions entered by the user on the front-end page;

[0033] The thought chain acquisition module is used to reason about the user problem and obtain the thought chain of the user problem, which includes multiple reasoning steps for solving the user problem.

[0034] The retrieval module is used to retrieve supporting text from a text database that matches each reasoning step in the thought chain;

[0035] The answer retrieval module is used to obtain the answer to the user question based on the user question and the retrieved supporting text;

[0036] The display module is used to display the answer on the front-end page.

[0037] Thirdly, embodiments of the present invention provide an electronic device, including: a processor, a memory, and a program stored in the memory and executable on the processor, wherein when the program is executed by the processor, it implements the steps of the question-and-answer method as described in the first aspect above.

[0038] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the question-and-answer method as described in the first aspect above.

[0039] Fifthly, embodiments of the present invention provide a computer program product, including computer instructions, which, when executed by a processor, implement the steps of the question-and-answer method as described in the first aspect above.

[0040] In this embodiment of the invention, reasoning is performed on the user's question to generate a thought chain for the user's question. Supporting text is then retrieved from the text database using the thought chain. That is, the retrieval part is broken down into thinking (obtaining the thought chain), then retrieving, then thinking again, and then answering. Since the semantics of the thought chain are closer to the supporting text than the semantics of the question, the semantic difference can be reduced during matching, improving the accuracy of text retrieval and thus improving the accuracy of answering the question. Attached Figure Description

[0041] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0042] Figure 1 This is a flowchart illustrating the question-and-answer method according to an embodiment of the present invention;

[0043] Figure 2 This is a schematic diagram of the structure of a reasoning model based on natural language understanding according to an embodiment of the present invention;

[0044] Figure 3 This is a schematic diagram of the structure of a reasoning model based on natural language understanding, according to another embodiment of the present invention.

[0045] Figure 4 This is a schematic diagram of the question-answering system according to an embodiment of the present invention;

[0046] Figure 5 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0047] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0048] Please refer to Figure 1 This invention provides a question-and-answer method, including:

[0049] Step S1: Obtain the user question entered by the user on the front-end page;

[0050] In this embodiment of the invention, the user question is a natural language question.

[0051] Step S2: Reason about the user problem to obtain the thought chain of the user problem, which includes multiple reasoning steps for solving the user problem;

[0052] The Chain of Thought (CoT) in this embodiment of the invention can also be called a multi-hop question retrieval chain, which is used to guide multi-hop retrieval.

[0053] In this embodiment of the invention, a large language model can be used to reason about the user's question and obtain the thought chain of the user's question.

[0054] Step S3: Retrieve supporting texts from the text database that match each reasoning step in the thought chain;

[0055] Step S4: Based on the user question and the retrieved supporting text, obtain the answer to the user question;

[0056] Step S5: Display the answer on the front-end page.

[0057] In this embodiment of the invention, reasoning is performed on the user's question to generate a thought chain for the user's question. Supporting text is then retrieved from the text database using the thought chain. That is, the retrieval part is broken down into thinking (obtaining the thought chain), then retrieving, then thinking again, and then answering. Since the semantics of the thought chain are closer to the supporting text than the semantics of the question, the semantic difference can be reduced during matching, improving the accuracy of text retrieval and thus improving the accuracy of answering the question.

[0058] The text database in this embodiment of the invention can be determined in the following ways.

[0059] Document Collection: The backend system first needs to collect various document resources. These documents can be files stored on a local server (such as TXT, PDF, Word, etc.) or web content crawled from the internet. For example, an enterprise knowledge base question-and-answer system would collect internal company rules and regulations, project documents, product manuals, etc.

[0060] Document Parsing: For documents of different formats, the backend system needs to use appropriate parsing tools to extract their content. For PDF documents, PDF parsing libraries (such as PyMuPDF in Python) can be used to extract the text content; for Word documents, libraries such as python-docx can be used to extract the text content. The extracted text content is usually converted into plain text format for subsequent processing.

[0061] Text cleaning: The extracted text may contain noisy information, such as headers, footers, page numbers, and extra spaces. The backend system needs to clean this text, removing irrelevant content and retaining only the useful main text. For example, regular expressions can be used to match and remove page number and header / footer formatting.

[0062] Document segmentation and sentence segmentation: The cleaned text is segmented and segmented into sentences according to certain rules. Segmentation can be done using paragraph marks, while sentence segmentation can be achieved by identifying sentence boundaries using Natural Language Processing (NLP) tools. This is because question-answering systems need to perform matching and understanding at the sentence or paragraph level when processing questions later. For example, NLP libraries such as NLTK (Natural Language Toolkit) or spaCy can be used to segment the text into sentences.

[0063] Through the above processing, a text database can be obtained.

[0064] The method for generating thought chains in the embodiments of the present invention will be described below.

[0065] The method for generating thought chains in this embodiment of the invention may include two approaches: zero-sample thought chains and few-sample prompts.

[0066] I. Zero-Sample Thinking Chain

[0067] Zero-Shot Chain of Thought (Zero-Shot-CoT) is a method for generating thought chains using large language models (LLMs) without requiring additional labeled data or training. It guides the large language model to automatically generate thought chains by incorporating specific instructions into question prompts. The core of this method lies in leveraging the pre-trained knowledge and generative capabilities of the large language model to directly generate thought chains on unseen tasks.

[0068] That is, reasoning is performed on the user's question to obtain the thought chain of the user's question, including:

[0069] Step S21: Based on the user question and the pre-set instructions, construct a first prompt, the instructions being used to guide the first inference model to generate multiple inference steps to solve the user question;

[0070] Constructing prompts is key to zero-shot thought chains. Prompts need to explicitly guide the first inference model in generating the reasoning process.

[0071] Step S22: Input the first prompt into the first inference model to obtain the thought chain of the user's question output by the first inference model.

[0072] The first inference model is a large-scale language model.

[0073] The following example illustrates this:

[0074] Example 1:

[0075] Question: "Are Scott Derrickson and Ed Wood of the same nationality?"

[0076] First hint: "To answer this question: "+Question+", what is the first step in thinking, and then use the result in the first step? What is the second step in thinking? You don't need to tell me the answer, just tell me the two steps."

[0077] Example 2:

[0078] Question: "What government position did the woman who played Corliss Archer in the movie 'Kiss and Tell' hold?"

[0079] First hint: "To answer this question: "+Question+", what is the first step in thinking, and then use the result in the first step? What is the second step in thinking? You don't need to tell me the answer, just tell me the two steps."

[0080] The first prompt is input into the first inference model, and the first inference model will generate a reasoning process based on the first prompt.

[0081] For example, for Example 1, the first inference model might generate the following:

[0082] The first step was to determine Scott Derrickson's nationality.

[0083] The second step is to determine Ed Wood's nationality.

[0084] For Example 2, the first inference model might generate the following:

[0085] The first step was to determine the actress who would play Corliss Archer in the film "Kiss and Tell".

[0086] The second step is to determine the actress's political or governmental position.

[0087] II. Few-Shot Prompting

[0088] Few-shot prompting is a method that guides a large language model (LLM) to complete a specific task by providing a small number of examples. The core of this approach lies in leveraging the pre-trained knowledge of the large language model, combined with a small amount of labeled data, to help the model better understand the task requirements, thereby generating more accurate and expected outputs. The key to few-shot prompting is demonstration. By showing the large language model a few examples relevant to the target task, it can learn the patterns and logic of the task, thus generating more accurate outputs on new inputs. This method does not require large amounts of labeled data or additional training of the large language model; instead, it guides the model through cleverly designed prompts.

[0089] That is, reasoning is performed on the user's question to obtain the thought chain of the user's question, including:

[0090] Step S21': Obtain examples related to the user question, the examples including user question examples and thought chains of the user question examples;

[0091] Choosing examples relevant to the target task is key. These examples should cover the main logic and patterns of the task.

[0092] In some embodiments, if multiple examples are available, a large language model can automatically select examples relevant to the user's question from among the multiple candidate examples. In other embodiments, a fixed set of examples may also be used.

[0093] Step S22': Based on the user question, the example, and the pre-set instructions, construct a second prompt, the instructions being used to guide the first inference model to generate multiple inference steps to solve the user question;

[0094] In some embodiments, the second prompt may optionally include the following parts:

[0095] Task Description: Briefly explain the nature of the task.

[0096] Example: Showing several examples related to the task.

[0097] Target input: The new input that the model needs to process (i.e., the user's question).

[0098] Instructions: Instructions that guide the model to generate output.

[0099] Step S23': Input the second prompt into the first inference model to obtain the thought chain of the user's question output by the first inference model.

[0100] The second prompt will be illustrated with an example below.

[0101] In one example, the second hint includes:

[0102] "To answer this question: Let's break it down into a few steps. What is the first step, and then, using the result of the first step, what is the second step? You don't need to tell me the answer, just tell me the two steps."

[0103] Here are some examples to help guide your thinking:

[0104] Example 1:

[0105] Question: "Are Scott Derrickson and Ed Wood of the same nationality?"

[0106] Step 1: Determine Scott Derrickson's nationality.

[0107] Step 2: Determine Ed Wood's nationality.

[0108] Example 2:

[0109] Question: "What government position did the woman who played Corliss Archer in the movie 'Kiss and Tell' hold?"

[0110] Step 1: Determine the actress to play Chris Archer in the film "Kiss and Tell".

[0111] Step 2: Determine the actress's political or governmental position.

[0112] Using these examples, think about the steps you would take to answer the initial question. Remember, I'm only asking for steps, not the final answer.

[0113] In this embodiment of the invention, the generated thought chain can also be cleaned, for example, by removing redundant text and symbols. For example, the cleaned thought chain might look like this: {"cot1": "Determine Scott Derrickson's nationality", "cot2": "Determine Ed Wood's nationality".}

[0114] It should be noted that in the above examples, the thought chain includes two reasoning steps. However, the embodiments of the present invention are not limited to this. Depending on the complexity of the problem, the thought chain may also include three or more reasoning steps.

[0115] In this embodiment of the invention, optionally, retrieving supporting text from a text database that matches each reasoning step in the thought chain includes:

[0116] Step S31: Based on the reasoning steps in the thought chain, construct a third prompt, which is used to guide the second reasoning model to retrieve supporting text from the text database that matches the reasoning steps of the thought chain;

[0117] Step S32: Input the third prompt into the second reasoning model to obtain the supporting text matching the reasoning steps of the thought chain output by the second reasoning model.

[0118] In this embodiment of the invention, the second inference model can be a large language model. The second inference model and the first inference model described above can be the same large language model, or they can be different large language models.

[0119] In this embodiment of the invention, the second inference model can also be trained in advance.

[0120] In some embodiments, optionally, the question-and-answer method further includes:

[0121] Step S01: Obtain a training set, which includes multiple sets of sample data. Each set of sample data includes: user question samples, reasoning steps in the thought chain samples of the user question samples, and text samples. The labels of the text samples are one of the following: supporting text corresponding to the reasoning steps, non-supporting text in the supporting text set of the user question samples, and irrelevant text.

[0122] Step S02: Input the sample data into the inference model to be trained, and obtain the inference result output by the inference model. The inference result is used to indicate whether the text sample in the sample data is supporting text that matches the inference step in the sample data. Determine the loss based on the inference result and the label of the text sample, and fine-tune the inference model based on the loss to obtain the trained second inference model.

[0123] The method for obtaining sample data in the embodiments of the present invention will be described below.

[0124] In some embodiments, optionally, the acquisition of sample data includes:

[0125] Step S011: Obtain the thought chain sample and supporting text set of the user's question sample;

[0126] Step S012: Based on each reasoning step in the thought chain sample and the supporting text in the supporting text set, construct a fourth hint, which is used to guide the third reasoning model to identify whether the supporting text matches the reasoning steps of the thought chain sample;

[0127] In this embodiment of the invention, optionally, the third inference model can be a large language model. The third inference model and the aforementioned first inference model and / or second inference model can be the same large language model, or they can be different large language models.

[0128] The following example illustrates the fourth tip.

[0129] Example 1: "Sentence A or B can recognize the content of sentence C. Sentence A is: '+cot1+' Sentence B is: '+cot2+' Sentence C is: '+supporting text+' Please help me determine whether sentence A can recognize sentence C, or whether sentence B can recognize sentence C."

[0130] cot1 and cot2 are two reasoning steps in the thought chain.

[0131] Example 2: "Given a sentence A, a sentence B, and a support sentence, determine which of sentence A and sentence B semantically matches the support sentence. Sentence A is: '+cot1+', sentence B is: '+cot2+', and the support sentence is: '+support text+'. If sentence A matches the support sentence, simply return the letter A; if sentence B matches the support sentence, simply return the letter B. You can only reply with either letter A or B."

[0132] Example 3: "Sentence A or B can identify the content in the supporting sentence. Sentence A is: "+cot1+", Sentence B is: "+cot2+", and the supporting sentence is: "+supporting text+". Help me determine whether sentence A or sentence B can identify the supporting sentence. If sentence A can identify the content of the supporting sentence, simply answer with the letter A. If sentence B can identify the content of the supporting sentence, simply answer with the letter B. Do not answer with other words or letters."

[0133] Example 4: "Given a sentence A, a sentence B, and a supporting sentence, determine which of the sentences A and B is a semantic match for the supporting sentence."

[0134] Example 5: "You are rating the following question:"

[0135] Sentence A: "+cot1+";

[0136] Sentence B: "+cot2+";

[0137] Supporting phrase: "+supporting text+";

[0138] If sentence A matches a supporting sentence, return A; if sentence B matches a supporting sentence, return B.

[0139] Reply: Your reply can only contain one letter, "A" or "B".

[0140] The following example illustrates this.

[0141] Example 1:

[0142] Sentence A: Identify a company or management group known for overseeing multiple shopping centers in the Peabody and Danvers areas.

[0143] Sentence B: "Considering the distance between them, verify whether the identified company simultaneously manages Peabody's North Shore Shopping Center and Danvers' Free Tree Shopping Center."

[0144] Supporting statement: "Free Tree Shopping Center is a shopping center in Danvers, Massachusetts, USA, in which Simon Property Group owns one-third."

[0145] Return: A.

[0146] Example 2:

[0147] Sentence A: Identify a company or management group known for overseeing multiple shopping centers in the Peabody and Danvers areas.

[0148] Sentence B: Considering the distance between them, verify whether the identified company simultaneously manages Peabody's North Shore Shopping Center and Danvers' Free Tree Shopping Center.

[0149] Supporting statement: The shopping center is currently managed by Simon Property Group and is less than a mile from Simon Free Tree Shopping Center.

[0150] Return: B.

[0151] Example 3:

[0152] Sentence A: Recognize Kerry Remsen and find out who her father is.

[0153] Sentence B: Based on the available information, determine the race of Kerry Remsen's father.

[0154] Supporting statement: She is the daughter of actor Bert Remsen and casting director Barbara Joyce Dodd, and the sister of Ann Remsen Manners. She has been married to Ron Cates since 2006.

[0155] Return: A.

[0156] Step S013: Input the fourth prompt into the third inference model to obtain the matching results of the supporting text output by the third inference model with each inference step in the thought chain sample;

[0157] Step S014: Determine the tag of the supporting text based on the matching result.

[0158] In some embodiments, one type of prompt (such as Example 1 in the fourth prompt example), one type of third inference model can be used, and the matching results are used as sample data to train the second inference model. In some embodiments, multiple types of prompts (such as Example 1, Example 2, and Example 3 in the fourth prompt example), or multiple types of third inference models can be used to obtain multiple matching results, and the common part of each matching result is taken as sample data to train the second inference model.

[0159] The defined training set may include the following sample data: {"Supporting text 1", "label": 1, "cot1", "question"}, {"Supporting text 2", "label": 1, "cot2", "question"}...{"Supporting text n", "label": 1, "cotn", "question"}, {"Non-supporting text 1", "label": 0, "cot1", "question"}...{"Non-supporting text m", "label": 0, "cot1", "question"}...

[0160] Wherein, Supporting Text 1 refers to the supporting text in the supporting text set of the user question sample that matches reasoning step 1 (cot1), Supporting Text 2 refers to the supporting text in the supporting text set of the user question sample that matches reasoning step 2 (cot2), Supporting Text n refers to the supporting text in the supporting text set of the user question sample that matches reasoning step n (cotn), Non-Supporting Text 1 refers to the text in the supporting text set of the user question sample that does not match reasoning step 1 (cot1), and Non-Supporting Text m refers to the text in the supporting text set of the user question sample that does not match reasoning step 1 (cot1).

[0161] Here, "label" refers to the label. A "label" value of 1 indicates that the text is supported, while a "label" value of 0 indicates that the text is not supported.

[0162] In this embodiment of the invention, the supporting text set and irrelevant text of the user question sample can be obtained in the following way: First, a large language model is used to perform preliminary screening on n documents in the text database to determine whether the documents contain supporting text. If they do, the search for supporting text continues; otherwise, the search for supporting text is not performed.

[0163] In this embodiment of the invention, the second reasoning model can be one of the following: a reasoning model based on natural language understanding, a reasoning model based on natural language generation, or a reasoning model based on knowledge graphs. The following describes each type of reasoning model.

[0164] I. Reasoning Model Based on Natural Language Understanding

[0165] Please refer to Figure 2 , Figure 2 This is a schematic diagram of the structure of a reasoning model based on natural language understanding according to an embodiment of the present invention. The reasoning model includes: a large language model, a fully connected layer, and an activation function.

[0166] The input to this natural language understanding-based inference model can be as follows:

[0167] The input is: question+[SEP]+cot1+[SEP]+text;

[0168] question + [SEP] + cot2 + [SEP] + text;

[0169] question+[SEP]+cotn+[SEP]+text.

[0170] [SEP] is the separator.

[0171] Large language models can be BERT or RoBERTa, which can be fine-tuned with all parameters. Alternatively, models like GPT4, deepseek, and qwen can be fine-tuned using LoRA.

[0172] Output: If the output is 1, then the text is supported.

[0173] If the output is 0, the text is not supported.

[0174] Please refer to Figure 3 , Figure 3 This is a schematic diagram of the structure of a reasoning model based on natural language understanding according to another embodiment of the present invention. The reasoning model includes: a large language model, multiple fully connected layers and multiple activation functions.

[0175] The input to this natural language understanding-based inference model can be as follows:

[0176] The input is: question+[SEP]+cot1+[SEP]+text;

[0177] question + [SEP] + cot2 + [SEP] + text;

[0178] question+[SEP]+cotn+[SEP]+text.

[0179] [SEP] is the separator.

[0180] Large language models can be BERT or RoBERTa, which can be fine-tuned with all parameters. Alternatively, models like GPT4, deepseek, and qwen can be fine-tuned using LoRA.

[0181] Output: If the output is 1, then the text is supported.

[0182] If the output is 0, the text is not supported.

[0183] II. Reasoning Model Based on Natural Language Generation

[0184] For a reasoning model based on natural language generation, the hints generated for this reasoning model can be as follows:

[0185] A) Hint: "The question is: "+question+", the clue is: "+cot+", the sentence is: "+sentence+", can this sentence serve as supporting text for answering the question?

[0186] B) Hint: "Which sentences in the following document can accomplish the first step? Give me the sentences, nothing else. Document: "+Context".

[0187] Inference models based on natural language generation can be large-scale language models such as GPT4, deepseek, and qwen.

[0188] III. Reasoning Model Based on Knowledge Graph

[0189] In this embodiment of the invention, the text retrieval method based on a knowledge graph-based reasoning model may include the following steps:

[0190] A) Transform document content into structured knowledge: First, perform entity recognition, relation extraction, and attribute annotation on the document to generate "entity-relation-entity" triples, and then store them in the knowledge graph.

[0191] B) Transform user questions and COT (thought chain) into graph queries: Perform semantic parsing (entity linking, relation mapping, intent classification) on natural language questions (user questions) to obtain query statements that can be executed on the knowledge graph (such as SPARQL or graph database queries).

[0192] C) Perform retrieval and reasoning on the graph: Use algorithms such as graph traversal, path search, and subgraph matching to find the subgraph or path that is most relevant to the user's question.

[0193] D) Convert search results back to natural language: Assemble the facts in the subgraph / path into a readable answer and return it to the user.

[0194] In this embodiment of the invention, after training the second inference model, the second inference model can also be tested.

[0195] The following describes the steps for obtaining the answer to the user question based on the user question and the retrieved supporting text.

[0196] In this embodiment of the invention, a large language model can be used to infer the answer to the user question based on the user question and the retrieved supporting text.

[0197] This large-scale language model can select pre-trained models suitable for document question answering tasks, such as chatGPT, qwen, and deepseek. These models are pre-trained on large-scale corpora and possess good language understanding and generation capabilities.

[0198] The large language model can be the same as or different from the first, second, or third inference model in the above embodiments.

[0199] In this embodiment of the invention, the pre-trained model used for reasoning answers can also be fine-tuned in advance. Specifically, the pre-trained model is fine-tuned according to the specific task and training set to improve its performance in document question answering tasks. Fine-tuning can be achieved by adding task-specific training data and optimization objectives. Alternatively, the pre-trained model can be used directly without fine-tuning.

[0200] Fine-tuning methods for pre-trained models can be categorized into full-parameter fine-tuning, partial-parameter fine-tuning, and parameter-efficient fine-tuning. Full-parameter fine-tuning achieves optimal performance by updating all parameters of the pre-trained model; partial-parameter fine-tuning updates only the top-level or specific layer parameters of the pre-trained model; and parameter-efficient fine-tuning achieves efficient adaptation by introducing additional structures or low-rank decompositions, including LoRA (Lower-Rank Matrix Factorization for weight updates), Adapter (inserting small trainable modules), Prompt Tuning (optimizing input cue vectors), and Prefix Tuning (optimizing Transformer key-value prefixes). These methods typically only require updating 0.1%-3% of the original model's parameters, significantly reducing memory usage and training time while maintaining good task performance, making them particularly suitable for the rapid adaptation of large language models with limited resources.

[0201] The following describes the front-end page display in the embodiments of the present invention.

[0202] I. User Interface Design

[0203] A clean and aesthetically pleasing layout: The front-end page needs a clean and clear layout to facilitate user input of questions and viewing of answers. In this embodiment of the invention, an input box is designed on the front-end page for users to input questions, with the answer display area below the input box. For example, in the front-end page of a question-and-answer system, the input box can be placed in the center of the page, with the background color coordinated with the overall style of the page. The font size and color of the input box are also carefully designed to allow users to comfortably input their questions.

[0204] Interactive element design: In addition to the input box and answer display area, there will be some interactive elements. For example, a "Submit" button, which the user clicks to send the question to the backend; a "Clear" button, used to clear the content in the input box; and a "History" button, which allows users to view previously entered questions and corresponding answers for easy review and comparison.

[0205] Multilingual support (if needed): If the question-and-answer system is geared towards multilingual users, the front-end page needs to support input and display in multiple languages. For example, users can choose to input questions in languages ​​such as Chinese or English, and the page will automatically switch to the corresponding language version based on the user's language settings. This requires cooperation between the back-end and front-end; the back-end must be able to handle questions in different languages, and the front-end must be able to correctly display answers in different languages.

[0206] II. Answer Display

[0207] Text Formatting: The answer text returned by the backend needs to be formatted and displayed on the frontend. For example, if the answer is a long paragraph, the frontend can break it into appropriate paragraph formats, set suitable line spacing and font size, so that users can read it clearly. For answers containing list information (such as "Li Bai's representative works include 'Bring in the Wine' and 'The Road to Shu is Hard'"), bullet points or numbering can be used for formatting.

[0208] Highlighting Key Information: To highlight key information in the answer, the front-end can highlight keywords or important information. For example, if the question is "What are Li Bai's representative works?", keywords such as "Li Bai" and "representative works" in the answer can be displayed in different colors or bolded, allowing users to quickly focus on the key content.

[0209] Multimedia content display: If the document contains multimedia content (such as images, videos, etc.), and the answer needs to display this content, the front-end page needs to be able to embed and display it correctly. For example, in a question-and-answer system about tourist attractions, when a user asks, "What does the Hall of Supreme Harmony in the Forbidden City look like?" the answer could include a picture of the Hall of Supreme Harmony. The front-end page would embed the picture into the answer display area and set an appropriate size and style.

[0210] III. User Feedback Collection

[0211] Satisfaction Survey: A user satisfaction survey button or icon can be placed next to the answer display area. Users can click to indicate their level of satisfaction with the answers, such as "Satisfied" or "Dissatisfied." The front-end sends user feedback to the back-end, which can then use this feedback to optimize the system, such as adjusting the answer retrieval strategy or improving the answer generation method.

[0212] Error Correction: If users find errors in the system's answers, they can provide a "Correct Error" button. After clicking, users can enter the correct answer or point out the error. This correction information will also be sent to the backend to help the system continuously improve its documentation and question-and-answer logic.

[0213] The question-and-answer system in this embodiment of the invention can also interact with users or other products.

[0214] I. Interaction with users

[0215] Real-time interaction: The question-and-answer system needs to be able to respond to user input in real time. While the user is typing a question, the front-end page can send the question to the back-end via asynchronous communication technologies (such as AJAX). The back-end processes the question and returns the answer to the front-end. The entire process should be completed in a short time, giving the user the feeling that the system is responsive in real time. For example, after a user enters a question, the system should return an answer within seconds.

[0216] Session Management: For question-and-answer scenarios that require contextual information, the system needs to manage sessions. For example, if a user first asks, "What is the architectural style of the Forbidden City?" and the system answers, and then the user asks, "What are the characteristics of its roof?", the system should be able to remember the previous question and answer, using "Forbidden City" as contextual information to answer the second question more accurately. The backend can track the user's session state using session identifiers (such as session IDs), and the frontend sends the session identifier to the backend with each request.

[0217] User guidance: If the user's input is unclear or the system cannot understand it, the front-end page can guide the user to re-enter the question through prompts. For example, the system can display prompts such as "Your question is not clear enough, please rephrase it" or "You may want to ask..." to help users better express their questions.

[0218] II. Interaction with other products

[0219] Data Sharing and Integration: The question-and-answer system can share and integrate data with other products. For example, it can integrate with an enterprise's internal knowledge management system, retrieving the latest document content from the knowledge management system and updating its own document library. Simultaneously, the question-and-answer system can also provide feedback on user question-and-answer records to the knowledge management system, helping it optimize its knowledge base. This integration can be achieved through application programming interfaces (APIs) for data transfer and interaction.

[0220] As a component of other products: Question-answering systems can also be embedded into other products as components. For example, in an online customer service system, the question-answering system can be embedded as a knowledge base question-answering module. When a user consults customer service, the customer service representative can use the question-answering system to quickly find the answer in relevant documents, improving customer service efficiency. In this case, the front-end page of the question-answering system can be integrated into the pages of other products through an embedded frame (such as an iframe), while the back-end interacts with the other products through a shared server or API.

[0221] Information push with other systems: The question-and-answer system can also interact with other systems to push information. For example, when the question-and-answer system updates document content or optimizes its question-and-answer strategy, it can push the update information to relevant systems (such as user notification systems) to remind users that the system has been updated and may provide better question-and-answer services. This information push can be implemented through message queues or event-driven mechanisms.

[0222] The question-and-answer system provides users with efficient and accurate question-and-answer services through complex backend processing, user-friendly frontend display, and efficient interaction with users and other products.

[0223] Please refer to Figure 4 This invention also provides a question-and-answer system 100, comprising:

[0224] The question acquisition module 101 is used to acquire user questions entered by the user on the front-end page;

[0225] The thought chain acquisition module 102 is used to reason about the user problem and obtain the thought chain of the user problem, wherein the thought chain includes multiple reasoning steps for solving the user problem.

[0226] The retrieval module 103 is used to retrieve supporting texts in a text database that match each reasoning step in the thought chain;

[0227] Answer acquisition module 104 is used to obtain the answer to the user question based on the user question and the retrieved supporting text;

[0228] Display module 105 is used to display the answer on the front-end page.

[0229] In this embodiment of the invention, reasoning is performed on the user's question to generate a thought chain for the user's question. Supporting text is then retrieved from the text database using the thought chain. That is, the retrieval part is broken down into thinking (obtaining the thought chain), then retrieving, then thinking again, and then answering. Since the semantics of the thought chain are closer to the supporting text than the semantics of the question, the semantic difference can be reduced during matching, improving the accuracy of text retrieval and thus improving the accuracy of answering the question.

[0230] Optionally, the thought chain acquisition module 102 is used to construct a first prompt based on the user question and a pre-set instruction, the instruction being used to guide a first inference model to generate multiple inference steps to solve the user question; and input the first prompt into the first inference model to obtain the thought chain of the user question output by the first inference model.

[0231] Optionally, the thought chain acquisition module 102 is used to acquire examples related to the user question, the examples including user question examples and thought chains of the user question examples; construct a second prompt based on the user question, the examples and pre-set instructions, the instructions being used to guide the first inference model to generate multiple inference steps to solve the user question; input the second prompt into the first inference model to obtain the thought chain of the user question output by the first inference model.

[0232] Optionally, the retrieval module 103 is used to construct a third prompt based on the reasoning steps in the thought chain. The third prompt is used to guide the second reasoning model to retrieve supporting text that matches the reasoning steps of the thought chain from the text database. The third prompt is then input into the second reasoning model to obtain the supporting text that matches the reasoning steps of the thought chain, output by the second reasoning model.

[0233] Optionally, the question-answering system 100 further includes:

[0234] A training module is used to acquire a training set, which includes multiple sets of sample data. Each set of sample data includes: user question samples, reasoning steps in the thought chain samples of the user question samples, and text samples. The labels of the text samples are one of the following: supporting text corresponding to the reasoning steps, non-supporting text in the supporting text set of the user question samples, and irrelevant text. The sample data is input into the inference model to be trained to obtain the inference result output by the inference model. The inference result is used to indicate whether the text samples in the sample data are supporting texts that match the reasoning steps in the sample data. The loss is determined based on the inference result and the labels of the text samples. The inference model is fine-tuned based on the loss to obtain a trained second inference model.

[0235] Optionally, the question-answering system 100 further includes:

[0236] The sample data acquisition module is used to acquire the thought chain sample and supporting text set of the user question sample; based on each reasoning step in the thought chain sample and the supporting text in the supporting text set, a fourth hint is constructed, which is used to guide the third reasoning model to identify whether the supporting text matches the reasoning steps of the thought chain sample; the fourth hint is input into the third reasoning model to obtain the matching result of the supporting text output by the third reasoning model and each reasoning step in the thought chain sample; and the label of the supporting text is determined based on the matching result.

[0237] Optionally, the second reasoning model is one of the following: a reasoning model based on natural language understanding, a reasoning model based on natural language generation, or a reasoning model based on knowledge graphs.

[0238] Please refer to Figure 5 The present invention also provides an electronic device 200, including a processor 201, a memory 202, and a computer program stored in the memory 202 and executable on the processor 201. When the computer program is executed by the processor 201, it implements the various processes of the above-described question-and-answer method embodiments and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0239] This invention also provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the various processes of the above-described question-and-answer method embodiments and achieves the same technical effects. To avoid repetition, it will not be described again here. The computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0240] This application also provides a computer program product, including computer instructions, which, when executed by a processor, implement the above-described... Figure 1 The various processes of the method embodiments shown can achieve the same technical effect, and will not be described again here to avoid repetition.

[0241] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0242] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0243] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of the present invention.

Claims

1. A question-and-answer method, characterized in that, include: Retrieve user questions entered by the user on the front-end page; Reasoning is performed on the user problem to obtain the thought chain of the user problem, which includes multiple reasoning steps for solving the user problem; Retrieve supporting text from the text database that matches each reasoning step in the thought chain; Based on the user's question and the retrieved supporting text, the answer to the user's question is obtained; The answer is displayed on the front-end page.

2. The method according to claim 1, characterized in that, Reasoning is performed on the user's question to obtain the thought chain of the user's question, including: Based on the user question and pre-set instructions, a first prompt is constructed, the instructions being used to guide the first inference model to generate multiple inference steps to solve the user question; The first prompt is input into the first inference model to obtain the thought chain of the user's question output by the first inference model.

3. The method according to claim 1, characterized in that, Reasoning is performed on the user's question to obtain the thought chain of the user's question, including: Obtain examples related to the user question, including examples of user questions and thought chains of the user question examples; Based on the user question, the example, and the pre-set instructions, a second prompt is constructed, the instructions being used to guide the first inference model to generate multiple inference steps to solve the user question; The second prompt is input into the first reasoning model to obtain the thought chain of the user's question output by the first reasoning model.

4. The method according to any one of claims 1-3, characterized in that, Retrieve supporting text from a text database that matches each reasoning step in the thought process, including: Based on the reasoning steps in the thought chain, a third prompt is constructed, which is used to guide the second reasoning model to retrieve supporting text from the text database that matches the reasoning steps of the thought chain; The third prompt is input into the second reasoning model to obtain the supporting text that matches the reasoning steps of the thought chain, output by the second reasoning model.

5. The method according to claim 4, characterized in that, Also includes: Obtain a training set, which includes multiple sets of sample data. Each set of sample data includes: user question samples, reasoning steps in the thought chain samples of the user question samples, and text samples. The labels of the text samples are one of the following: supporting text corresponding to the reasoning steps, non-supporting text in the supporting text set of the user question samples, and irrelevant text. The sample data is input into the inference model to be trained to obtain the inference result output by the inference model. The inference result is used to indicate whether the text sample in the sample data is supporting text that matches the inference step in the sample data. The loss is determined based on the inference result and the label of the text sample. The inference model is fine-tuned based on the loss to obtain the trained second inference model.

6. The method according to claim 5, characterized in that, Also includes: Obtain sample data; The acquisition of sample data includes: Obtain a thought chain sample and supporting text set for user question samples; Based on each reasoning step in the thought chain sample and the supporting text in the supporting text set, a fourth prompt is constructed. The fourth prompt is used to guide the third reasoning model to identify whether the supporting text matches the reasoning steps of the thought chain sample. The fourth prompt is input into the third reasoning model to obtain the matching results between the supporting text output by the third reasoning model and each reasoning step in the thought chain sample; The tags for the supporting text are determined based on the matching results.

7. The method according to claim 4, characterized in that, The second reasoning model is one of the following: a reasoning model based on natural language understanding, a reasoning model based on natural language generation, or a reasoning model based on knowledge graphs.

8. A question-and-answer system, characterized in that, include: The question retrieval module is used to retrieve user questions entered by the user on the front-end page; The thought chain acquisition module is used to reason about the user problem and obtain the thought chain of the user problem, which includes multiple reasoning steps for solving the user problem. The retrieval module is used to retrieve supporting text from a text database that matches each reasoning step in the thought chain; The answer retrieval module is used to obtain the answer to the user question based on the user question and the retrieved supporting text; The display module is used to display the answer on the front-end page.

9. An electronic device, characterized in that, include: A processor, a memory, and a program stored in the memory and executable on the processor, wherein the program, when executed by the processor, implements the steps of the question-and-answer method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the question-and-answer method as described in any one of claims 1 to 7.

11. A computer program product, characterized in that, It includes computer instructions that, when executed by a processor, implement the steps of the question-and-answer method as described in any one of claims 1 to 7.

Images