Question Answering Method, Device, Electronic Device and Medium Based on Large Language Model

Abstract

This specification provides a question-answering method, apparatus, electronic device, and medium based on a large language model. The method includes: performing a knowledge base retrieval on the original question to obtain multiple first retrieval results and the retrieval scores of the multiple first retrieval results; in the case where there are first retrieval results with retrieval scores less than the score threshold among the multiple first retrieval results, using the large language model to determine whether the original question is a simple question; if the original question is a simple question, performing query expansion processing on the original question to obtain a new question; performing a knowledge base retrieval on the new question to obtain second retrieval results; and determining an answer message based on the second retrieval results and the first retrieval results among the multiple first retrieval results with retrieval scores greater than or equal to the score threshold.

Description

Technical Field

[0001] This specification relates to the field of artificial intelligence technology, and particularly to a question and answer method, apparatus, electronic device, and medium based on a large language model. Background Art

[0002] RAG (Retrieval Augmented Generation) is a method that combines a very large pre-trained language model with an external retrieval or search mechanism. The core idea of RAG is to allow a generative AI model to extract information from a large number of documents during the generation process to generate more accurate and rich text. With the large increase in the volume and complexity of document corpora, it has become increasingly challenging to improve the accuracy of RAG. Summary of the Invention

[0003] To overcome the problems existing in the related art, this specification provides a question and answer method, apparatus, electronic device, and medium based on a large language model.

[0004] According to the first aspect of the embodiments of this specification, a question and answer method based on a large language model is provided. The method includes: performing a knowledge base retrieval on an original question to obtain a plurality of first retrieval results and the retrieval scores of the plurality of first retrieval results; in the case where there is a first retrieval result with a retrieval score less than a score threshold among the plurality of first retrieval results, using the large language model to determine whether the original question is a simple question; if the original question is a simple question, performing a query expansion process on the original question to obtain a new question; performing a knowledge base retrieval on the new question to obtain a second retrieval result; and determining an answer message according to the second retrieval result and the first retrieval results among the plurality of first retrieval results with a retrieval score greater than or equal to the score threshold.

[0005] According to the second aspect of the embodiments of this specification, a question and answer apparatus based on a large language model is provided, including: a first retrieval module for performing a knowledge base retrieval on an original question to obtain a plurality of first retrieval results and the retrieval scores of the plurality of first retrieval results; a question analysis module for, in the case where there is a first retrieval result with a retrieval score less than a score threshold among the plurality of first retrieval results, using the large language model to determine whether the original question is a simple question; a query expansion module for, if the original question is a simple question, performing a query expansion process on the original question to obtain a new question; a second retrieval module for performing a knowledge base retrieval on the new question to obtain a second retrieval result; and a first answer determination module for determining an answer message according to the second retrieval result and the first retrieval results among the plurality of first retrieval results with a retrieval score greater than or equal to the score threshold.

[0006] According to a third aspect of the embodiments of the present specification, there is provided an electronic device, including:

[0007] A processor;

[0008] A memory for storing instructions executable by the processor;

[0009] Wherein, the processor is configured to execute the question-and-answer method based on the large language model according to the first aspect or any corresponding implementation manner thereof.

[0010] According to a fourth aspect of the embodiments of the present specification, there is provided a computer-readable storage medium, on which computer instructions are stored, and the computer instructions are used to cause a computer to execute the question-and-answer method based on the large language model according to the first aspect or any corresponding implementation manner thereof.

[0011] The technical solutions provided by the embodiments of the present specification may include the following beneficial effects:

[0012] In the embodiments of the present specification, by retaining the retrieval results with higher retrieval scores and then performing query expansion or multi-step reasoning on the retrieval results with lower retrieval scores to make them return more accurate retrieval results, the influence of the retrieval results on the large language model can be reduced, and the accuracy of the results returned by the large language model can be improved.

[0013] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present specification. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The drawings herein are incorporated into the specification and constitute a part of the specification, showing the embodiments consistent with the present specification, and are used together with the specification to explain the principles of the present specification.

[0015] Figure 1 is a schematic diagram of a system architecture shown according to an exemplary embodiment of the present specification.

[0016] Figure 2 is a flowchart of a question-and-answer method based on a large language model shown according to an exemplary embodiment of the present specification.

[0017] Figure 3 is a flowchart of another question-and-answer method based on a large language model shown according to an exemplary embodiment of the present specification.

[0018] Figure 4 is a hardware structure diagram of a computer device where the question-and-answer device based on the large language model of the embodiments of the present specification is located.

[0019] Figure 5It is a block diagram of a question-answering device based on a large language model shown in this specification according to an exemplary embodiment. Detailed implementation manners

[0020] Here, the exemplary embodiments will be described in detail, and the examples are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with this specification. On the contrary, they are merely examples of devices and methods consistent with some aspects of this specification as detailed in the appended claims.

[0021] The terms used in this specification are only for the purpose of describing specific embodiments and are not intended to limit this specification. The singular forms "a", "an", and "the" used in this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0022] It should be understood that although the terms first, second, third, etc. may be used in this specification to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this specification, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "when" or "while" or "in response to determining".

[0023] Next, the embodiments of this specification will be described in detail.

[0024] The following combines Figure 1 to describe the system architecture to which the question-answering method and device based on a large language model can be applied in the embodiments of this specification. It should be noted that Figure 1 The illustration is only an example of the system architecture to which the embodiments of this specification can be applied to help those skilled in the art understand the technical content of this specification, but it does not mean that the embodiments of this specification cannot be used in other devices, systems, environments, or scenarios.

[0025] Figure 1 It is a schematic diagram of the system architecture shown in this specification according to an exemplary embodiment.

[0026] As Figure 1As shown, the system architecture may include, for example, terminal devices, a network, and a server. The network is used to provide a medium for communication links between terminal devices and the server. The network may include various connection types, such as wired and / or wireless communication links, etc.

[0027] Users can use terminal devices to interact with the server through the network to receive or send messages, etc. Various communication client applications can be installed on the terminal devices, such as web browser applications, search applications, instant messaging tools, email clients, and / or social platform software, etc.

[0028] The terminal device can be various electronic devices with a display screen and supporting web browsing, including but not limited to smartphones, tablets, laptop computers, and desktop computers, etc.

[0029] The server can be a server that provides various services, such as a background management server that supports the content browsed by users using terminal devices. The background management server can analyze and process data such as received user requests, and feedback the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0030] Next, the question-answering method based on a large language model provided in the embodiments of this specification will be described in detail. As Figure 2 shown, Figure 2 is a flowchart of a question-answering method based on a large language model shown in this specification according to an exemplary embodiment. This method can be applied to, for example, a server. The question-answering method based on a large language model provided in the embodiments of this specification may include the following steps.

[0031] In step 210, perform a knowledge base retrieval on the original question to obtain multiple first retrieval results and the retrieval scores of the multiple first retrieval results.

[0032] According to the embodiments of this specification, the retrieval score can be used to represent the degree of association between the first retrieval result and the original question.

[0033] According to the embodiments of the present specification, for example, keywords in the original question can be determined. Based on the keywords, a search is performed in the knowledge base to obtain at least one first retrieved document and the retrieval scores of at least one first retrieved document. The similarity between each document in the knowledge base and the original question is calculated. Based on the similarity, at least one second retrieved document is determined from the knowledge base. The first retrieval result includes the first retrieved documents and the second retrieved documents. Then, at least one first retrieved document, the retrieval scores of at least one first retrieved document, at least one second retrieved document, and the similarity of at least one second retrieved document are input into a re-ranking model to obtain the retrieval scores of each retrieved document. The re-ranking model can be used to combine the retrieval scores and similarities to re-score and re-rank the retrieved documents to obtain more accurate retrieval scores for each retrieved document.

[0034] In step 220, in the case where there are first retrieval results with retrieval scores less than the score threshold among multiple first retrieval results, a large language model is used to determine whether the original question is a simple question. According to the embodiments of the present specification, a retrieval score less than the score threshold indicates a low relevance between the retrieval result and the question. The score threshold can be set according to actual needs. Exemplarily, in this embodiment, the average value of the retrieval scores of all retrieval results can be calculated as the score threshold.

[0035] According to the embodiments of the present specification, for example, a large language model can be used to analyze whether the original question contains multi-faceted information and whether it is a comparison type question. If the original question does not contain multi-faceted information and the original question is not a comparison type question, then the original question is determined to be a simple question; otherwise, it is determined that the original question is not a simple question.

[0036] In step 230, if the original question is a simple question, query expansion processing is performed on the original question to obtain a new question.

[0037] According to the embodiments of the present specification, for example, a large language model can be used to determine the domain information of the original question, and based on the domain information, pseudo-documents related to the original question are generated. Then, the pseudo-documents and the original question are merged to obtain a new question.

[0038] For example, the original question can be "how to improve sleep". A large language model can be used to analyze the original question and generate pseudo-documents in the relevant field, such as "To improve sleep, maintain regular sleep hours", "Avoid caffeine", and "Reduce the use of electronic devices, etc.", and then the original question and the pseudo-documents are concatenated to obtain a new question.

[0039] In step 240, a knowledge base search is performed on the new question to obtain a second retrieval result.

[0040] According to the embodiments of this specification, the steps of performing a knowledge base retrieval for a new problem can, for example, refer to the steps of performing a knowledge base retrieval for the original problem, which will not be elaborated here.

[0041] In step 250, based on the second retrieval result and the first retrieval results among the multiple first retrieval results whose retrieval scores are greater than or equal to the score threshold, determine the answer information.

[0042] According to the embodiments of this specification, for example, a re-ranking model can be used to adjust the scores of the second retrieval result and the first retrieval results among the multiple first retrieval results whose retrieval scores are greater than or equal to the score threshold to obtain the adjusted scores. According to the adjusted scores, select the top k retrieval results with the highest scores (topK). Wherein, k is a positive integer and can be set according to actual needs. Then a large language model can be used to generate the answer information based on these k retrieval results.

[0043] According to the embodiments of this specification, by retaining the retrieval results with higher retrieval scores and then performing query expansion or multi-step reasoning on the retrieval results with lower retrieval scores to make them return more accurate retrieval results, the influence of the retrieval results on the large language model can be reduced, and the accuracy of the results returned by the large language model can be improved.

[0044] Optionally, if the original problem is not a simple problem, use the large language model to disassemble the original problem to obtain a problem chain, and the problem chain includes multiple sub-questions. Use the large language model to gradually reason about each question in the problem chain to obtain the third retrieval result. Then, based on the third retrieval result and the first retrieval results among the multiple first retrieval results whose retrieval scores are greater than or equal to the score threshold, determine the answer information.

[0045] For example, the original problem is: Xiaoming needs to queue for 3 minutes each time he plays bumper cars, and then he can drive the bumper cars for 5 minutes. The amusement park will close in 30 minutes. How many times can Xiaoming play before it closes?

[0046] Stage 1: The large language model decomposes the original problem into sub-problems.

[0047] In this embodiment, the large language model can decompose the complex problem into smaller sub-problems: To answer "How many times can Xiaoming play before it closes?", it is necessary to first solve the problem of "How long does it take to play bumper cars each time?".

[0048] Stage 2: The large language model sequentially solves the sub-problems for chain reasoning.

[0049] In this embodiment, for sub-question 1, the reasoning process is as follows:

[0050] Q: How long does it take to play bumper cars each time? A: It takes Xiaoming 3 minutes to queue up and 5 minutes to drive the bumper cars. So it takes 8 minutes to play bumper cars each time.

[0051] For sub-question 2, the reasoning process is as follows:

[0052] Q: How many times can Xiaoming play before it closes? A: The amusement park will close in 30 minutes and it takes 8 minutes to play bumper cars each time. So Xiaoming can play 30 ÷ 8 = 3.75 times. Since he can't drive for a fraction of a time, Xiaoming can play 3 times before it closes.

[0053] According to the embodiments of this specification, for example, a re-ranking model can be used to adjust the scores of the third retrieval result and the first retrieval results among multiple first retrieval results whose retrieval scores are greater than or equal to the score threshold to obtain the adjusted scores. According to the adjusted scores, the j retrieval results with the highest scores are selected. Where j is a positive integer and can be set according to actual needs. Then a large language model can be used to generate answer information based on these j retrieval results.

[0054] According to the embodiments of this specification, by performing query expansion and multi-step reasoning on the question, the retrieval of the document content can be effectively enhanced to make it return multi-channel recall content. In addition, after the re-ranking model returns multiple recall texts with the highest scores, the large language model can better understand the needs of the user's question, enabling the large language model to retrieve the correct content from a large number of documents.

[0055] According to the embodiments of this specification, the large language model can be instructed to determine whether the original question is a simple question according to the routing selection prompt (prompt word).

[0056] Exemplarily, in this embodiment, the routing selection prompt may include:

[0057] Please analyze and judge which type of question the question raised by the user belongs to according to the following rules and output it in strict accordance with the specified format. The judgment rules are as follows:

[0058] ① If the user's question involves comparison, it is a complex question; otherwise, it is a simple question.

[0059] ② If the user's question can be split into multiple sub-questions, it is a complex question; otherwise, it is a simple question.

[0060] ③ If it does not fall into the above two judgments, it is a simple question.

[0061] For example, the original question is "Now I want to choose a router with high cost performance. Should I choose Router A or Router B?"

[0062] The output of the large language model is: Complex question

[0063] According to the embodiments of this specification, the large language model can be instructed by the problem decomposition prompt to decompose the original problem.

[0064] Exemplarily, in this embodiment, the problem decomposition prompt may include:

[0065] Please follow the following splitting rules according to the problem raised by the user and output in the specified format strictly.

[0066] The splitting rules are as follows:

[0067] ① Your purpose is to solve the problem that the complex problem of the user cannot be retrieved from the knowledge base well, and you can return the complete information.

[0068] ② Ensure that each sub-problem split out is complete and does not generate information unrelated to the original problem, and the semantics is as close as possible to the original problem.

[0069] ③ Ensure that each sub-problem is a simple problem and at most no more than 3 are generated, which can meet the needs of chain reasoning.

[0070] For example, the original problem is "Which is faster, taking the high-speed rail or the plane from Xi'an to Chongqing?"

[0071] The output of the large language model:

[0072] Split problem 1: How long does it take to take the high-speed rail from Xi'an to Chongqing?

[0073] Split problem 2: How long does it take to take the plane from Xi'an to Chongqing?

[0074] According to the embodiments of this specification, the large language model can be instructed by the final reasoning prompt to generate answer information.

[0075] Exemplarily, in this embodiment, the final reasoning prompt may include:

[0076] Please follow the following summarization principles according to the problem raised by the user: and output in the specified format strictly.

[0077] The summarization principles are as follows:

[0078] ① Extract the key points and main concepts related to the theme from the given information.

[0079] ② Based on this information, conduct logical reasoning and summarize the comprehensive conclusion of the theme.

[0080] ③ Ensure that your answer is simple and clear and does not contain the semantics before the prompt information.

[0081] ④ Finally, please present your reasoning summary in a professional and objective tone.

[0082] Case: The user's question is "When traveling from Xi'an to Chongqing, is it faster to take the high-speed train or the plane?"

[0083] TopK information: {"It takes about 4 hours to take the high-speed train from Xi'an to Chongqing", "It takes about 2 hours to take the plane from Xi'an to Chongqing"}

[0084] Final reasoning: According to the known information, it takes 4 hours to take the high-speed train from Xi'an to Chongqing, which is longer than the 2 hours it takes to take the plane. Therefore, it is faster to take the plane.

[0085] It should be noted that the above prompt cases are for reference only, and the prompt can be changed according to specific scenarios.

[0086] As Figure 3 shown, it is a flowchart of another question-answering method based on a large language model shown according to an exemplary embodiment. The question-answering method based on the large language model may include the following steps:

[0087] Step 301: The user first asks a question, that is, the original question. In this embodiment, after splitting and vectorizing the text of the question, a question vector can be obtained. According to the question vector, vector similarity and keyword retrieval are performed, and a sentence window is used for location retrieval and context splicing to obtain text information, that is, the retrieval result. In this embodiment, ES (Elasticsearch) can be used for keyword retrieval. Elasticsearch is a distributed search and analysis engine.

[0088] Optionally, in this embodiment, 3 pieces of text information can be recalled by default. After calculating the vector similarity and keyword retrieval, 3 pieces of text information and the recall score of each piece of text information are recalled, and then the retrieval score returned after passing through the re-ranking model is obtained. It is judged whether the retrieval scores of these 3 recalled texts are greater than the score threshold. If so, the current recalled text information is retained; otherwise, step 302 is performed. Through this step, some recalled information with low relevance can be eliminated. Exemplarily, in this embodiment, the average value calculation function avg() can be used to calculate the average value of the retrieval scores of all retrieval results as the score threshold.

[0089] Step 302: When the retrieval score is less than the score threshold, the current recalled text is no longer used. By designing a function calling (functional callback function) through the LLM and prompt (prompt word), and through the optimization of the prompt and output format limitation, the LLM is allowed to judge for AI semantic routing to determine whether the current original question belongs to a simple question (referring to a question with only one simple interrogative sentence, not involving comparison and multiple interrogative sentences). If it belongs to a simple question, step 303 is performed; otherwise, step 304 is performed.

[0090] Step 303: When the question is a simple one, it is necessary to prove that the original question fails to retrieve good content, and query expansion of the original question is required.

[0091] Optionally, in this embodiment, the Query2doc (query-to-document) method can be used to generate some pseudo-documents using the LLM, and then these pseudo-documents are merged with the original question to form a new question. The content of the pseudo-documents needs to be restricted within the current retrieval scope, and the generated content is returned as new retrieved content.

[0092] Step 304: When the original question is a complex one, multi-step reasoning can be performed on the original question.

[0093] Exemplarily, in this embodiment, the LLM and prompt can be used to disassemble the question, and then the chain reasoning method is used to gradually solve the problem. Each question is answered based on the previous question. The prompt clearly states that the answer is only based on the search results and no other information should be added. Optionally, according to the scenario requirements, some subjectivity or the LLM's understanding of knowledge can be appropriately incorporated into the LLM's answer.

[0094] Step 305: Finally, the query expansion result or the result after the chain reasoning of the complex question and the retrieved content with a retrieval score greater than or equal to the score threshold before are passed through a re-ranking model to re-adjust the scores retrieved from the knowledge base according to the true intention of the user's question, and finally the k retrieved texts with the highest similarity are returned. Thus, the returned result can be made more realistic.

[0095] Step 306: Finally, the k retrieved texts are handed over to the LLM, along with the knowledge base reasoning summary prompt. For example, the prompt can be set as: "You are an intelligent customer service. Your goal is to provide accurate information and help the questioner solve the problem as much as possible. Known information: xxx, please answer according to the above information. Answer based on the existing knowledge without adding any guesses, and answer in Chinese. If you cannot answer according to the known content, please answer 'The question cannot be answered based on the known information'." Finally, the answer that the user actually wants to get is returned after the LLM reasoning summary.

[0096] Corresponding to the embodiment of the foregoing method, this specification also provides an embodiment of a question-answering device based on a large language model and a terminal to which it is applied.

[0097] Embodiments of the question - answering device based on large - language models in this specification can be applied to computer devices, such as servers or terminal devices. The device embodiments can be implemented through software, or through hardware, or a combination of software and hardware. Taking software implementation as an example, as a logically defined device, it is formed by the processor in which it is located reading the corresponding computer program instructions in the non - volatile memory into the memory for operation. From a hardware perspective, as Figure 4 shown, it is a hardware structure diagram of the computer device where the question - answering device based on large - language models in the embodiments of this specification is located. In addition to Figure 4 the processor 410, memory 430, network interface 420, and non - volatile memory 440 shown, for the server or electronic device where the device 431 is located in the embodiments, usually according to the actual functions of this computer device, it may also include other hardware, which will not be elaborated here.

[0098] As Figure 5 shown, Figure 5 is a block diagram of a question - answering device based on large - language models shown in an exemplary embodiment of this specification. The device includes:

[0099] A first retrieval module 510, configured to perform a knowledge - base retrieval on the original question to obtain a plurality of first retrieval results and the retrieval scores of the plurality of first retrieval results;

[0100] A question analysis module 520, configured to use a large - language model to determine whether the original question is a simple question in the case where there are first retrieval results with retrieval scores less than the score threshold among the plurality of first retrieval results;

[0101] A query expansion module 530, configured to perform query expansion processing on the original question to obtain a new question if the original question is a simple question;

[0102] A second retrieval module 540, configured to perform a knowledge - base retrieval on the new question to obtain a second retrieval result;

[0103] A first answer determination module 550, configured to determine answer information based on the second retrieval result and the first retrieval results among the plurality of first retrieval results with retrieval scores greater than or equal to the score threshold.

[0104] Optionally, the device may further include:

[0105] A disassembling module, configured to use a large - language model to perform disassembling processing on the original question to obtain a question chain if the original question is not a simple question, where the question chain includes a plurality of sub - questions;

[0106] An inference module, configured to use a large - language model to gradually perform inference on each question in the question chain to obtain a third retrieval result;

[0107] The second answer determination module is configured to determine answer information based on the third search result and a first search result having a search score greater than or equal to a score threshold among the plurality of first search results.

[0108] Optionally, the first search result includes a first recalled document and a second recalled document; and the first search module may include:

[0109] Keyword determination submodule, used to determine the keywords in the original question;

[0110] A keyword retrieval submodule, configured to search the knowledge base based on the keyword to obtain at least one first recalled document and a recall score of at least one first recalled document;

[0111] Similarity calculation submodule, used to calculate the similarity between each document in the knowledge base and the original question;

[0112] A similarity retrieval submodule, configured to determine at least one second recalled document from the knowledge base based on the similarity;

[0113] The score determination submodule is used to input at least one first recalled document, the recall score of at least one first recalled document, at least one second recalled document and the similarity of at least one second recalled document into the re-ranking model to obtain a retrieval score.

[0114] Optionally, the problem analysis module may include:

[0115] The analysis submodule uses a large language model to analyze whether the original question contains multiple aspects of information and whether it is a comparative question.

[0116] The judgment submodule is used to determine that the original question is a simple question if the original question does not contain multiple information and is not a comparison question; otherwise, the original question is determined to be not a simple question.

[0117] Optionally, the query expansion module may include:

[0118] The domain determination submodule is used to use the large language model to determine the domain information of the original question and generate pseudo documents related to the original question based on the domain information;

[0119] The merging submodule is used to merge the pseudo document with the original question to obtain a new question.

[0120] According to the embodiments of this specification, by retaining retrieval results with higher retrieval scores and then performing query expansion or multi-step reasoning on retrieval results with lower retrieval scores, more accurate retrieval results can be returned, thereby reducing the influence of the retrieval results on the large language model and improving the accuracy of the results returned by the large language model.

[0121] Correspondingly, this specification also provides an electronic device, which includes a processor; a memory for storing instructions executable by the processor; wherein, the processor is configured to: perform a knowledge base retrieval on the original question to obtain a plurality of first retrieval results and the retrieval scores of the plurality of first retrieval results; in the case that there are first retrieval results with retrieval scores less than the score threshold among the plurality of first retrieval results, use a large language model to determine whether the original question is a simple question; if the original question is a simple question, perform query expansion processing on the original question to obtain a new question; perform a knowledge base retrieval on the new question to obtain a second retrieval result; and determine an answer message based on the second retrieval result and the first retrieval results among the plurality of first retrieval results with retrieval scores greater than or equal to the score threshold.

[0122] For the implementation processes of the functions and roles of each module in the above device, please refer to the implementation processes of the corresponding steps in the above method for details, which will not be elaborated here.

[0123] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the partial descriptions of the method embodiments. The device embodiments described above are only illustrative. The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed to multiple network modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution in this specification. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0124] The above describes specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be executed in a different order than in the embodiments and still achieve the desired results. Additionally, the processes depicted in the drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0125] Those skilled in the art will readily conceive of other embodiments of this specification after considering the specification and practicing the invention claimed herein. This specification is intended to cover any variations, uses, or adaptations of this specification, which follow the general principles of this specification and include common general knowledge or conventional technical means in the technical field not claimed in this application. The specification and embodiments are only regarded as exemplary, and the true scope and spirit of this specification are pointed out by the following claims.

[0126] It should be understood that the present specification is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present specification is only limited by the appended claims.

[0127] The above are only the preferred embodiments of the present specification and are not intended to limit the present specification. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present specification shall be included within the scope of protection of the present specification.

Claims

1. A question-answering method based on large language models, characterized in that, The method includes: Performing a knowledge base retrieval on the original question to obtain multiple first retrieval results and the retrieval scores of the multiple first retrieval results; When there is a first retrieval result with a retrieval score less than the score threshold among the multiple first retrieval results, using a large language model to determine whether the original question is a simple question; If the original question is a simple question, performing query expansion processing on the original question to obtain a new question; Performing a knowledge base retrieval on the new question to obtain second retrieval results; Determining answer information based on the second retrieval results and the first retrieval results among the multiple first retrieval results with retrieval scores greater than or equal to the score threshold; If the original question is not a simple question, using a large language model to disassemble the original question to obtain a question chain, where the question chain includes multiple sub-questions; Using a large language model to gradually reason about each question in the question chain to obtain third retrieval results; Determining answer information based on the third retrieval results and the first retrieval results among the multiple first retrieval results with retrieval scores greater than or equal to the score threshold.

2. The method according to claim 1, characterized in that, The first retrieval results include a first recalled document and a second recalled document; the performing a knowledge base retrieval on the original question to obtain first retrieval results and retrieval scores includes: Determining the keywords in the original question; Performing a retrieval in the knowledge base according to the keywords to obtain at least one first recalled document and the recall scores of the at least one first recalled document; Calculating the similarity between each document in the knowledge base and the original question; Determining at least one second recalled document from the knowledge base according to the similarity; Inputting the at least one first recalled document, the recall scores of the at least one first recalled document, the at least one second recalled document, and the similarity of the at least one second recalled document into a re-ranking model to obtain a retrieval score.

3. The method according to claim 1, wherein The using a large language model to determine whether the original question is a simple question includes: Using a large language model to analyze whether the original question contains multi-faceted information and whether it is a comparison type question; If the original question does not contain multi-faceted information and the original question is not a comparison type question, determining that the original question is a simple question, otherwise, determining that the original question is not a simple question.

4. The method according to claim 1, characterized in that, The performing query expansion processing on the original question to obtain a new question includes: Using a large language model to determine the domain information of the original question and generating a pseudo-document related to the original question according to the domain information; Merging the pseudo-document with the original question to obtain the new question.

5. A question and answer device based on a large language model, characterized in that, The device includes: A first retrieval module for performing a knowledge base retrieval on the original question to obtain multiple first retrieval results and the retrieval scores of the multiple first retrieval results; A question analysis module for using a large language model to determine whether the original question is a simple question when there is a first retrieval result with a retrieval score less than the score threshold among the multiple first retrieval results; A query expansion module, configured to perform query expansion processing on the original question to obtain a new question if the original question is a simple question; A second retrieval module, configured to perform knowledge base retrieval on the new question to obtain a second retrieval result; A first answer determination module, configured to determine an answer information according to the second retrieval result and the first retrieval results in the multiple first retrieval results whose retrieval scores are greater than or equal to a score threshold; A disassembling module, configured to use a large language model to perform disassembling processing on the original question to obtain a question chain if the original question is not a simple question, where the question chain includes multiple sub-questions; An inference module, configured to use a large language model to gradually perform inference on each question in the question chain to obtain a third retrieval result; A second answer determination module, configured to determine an answer information according to the third retrieval result and the first retrieval results in the multiple first retrieval results whose retrieval scores are greater than or equal to a score threshold; 6. The device according to claim 5, characterized in that The first retrieval result includes a first recalled document and a second recalled document; the first retrieval module includes: A keyword determination sub-module, configured to determine keywords in the original question; A keyword retrieval sub-module, configured to perform retrieval in a knowledge base according to the keywords to obtain at least one first recalled document and the recall scores of the at least one first recalled document; A similarity calculation sub-module, configured to calculate the similarity between each document in the knowledge base and the original question; A similarity retrieval sub-module, configured to determine at least one second recalled document from the knowledge base according to the similarity; A score determination sub-module, configured to input the at least one first recalled document, the recall scores of the at least one first recalled document, the at least one second recalled document, and the similarity of the at least one second recalled document into a re-ranking model to obtain a retrieval score; 7. The device according to claim 5, wherein The question analysis module includes: An analysis sub-module, configured to use a large language model to analyze whether the original question contains multi-faceted information and whether it is a comparison type question; A judgment sub-module, configured to determine that the original question is a simple question if the original question does not contain multi-faceted information and the original question is not a comparison type question, otherwise, determine that the original question is not a simple question; 8. The device according to claim 5, characterized in that The query expansion module includes: A domain determination sub-module, configured to use a large language model to determine the domain information of the original question and generate a pseudo-document related to the original question according to the domain information; A merging sub-module, configured to merge the pseudo-document with the original question to obtain the new question; 9. An electronic device, comprising: A processor; A memory for storing instructions executable by the processor; Wherein, the processor is configured to execute the method according to any one of claims 1 to 4; 10. A computer-readable storage medium, characterized in that, Computer instructions are stored on the computer-readable storage medium, and the computer instructions are used to cause a computer to execute the method according to any one of claims 1 to 4.

Images