Question and answer method and device based on large language model, electronic equipment and medium

Abstract

The invention provides a question answering method and device based on a large language model, electronic equipment and a medium. The method comprises the following steps: performing knowledge base retrieval on an original question to obtain a plurality of first retrieval results and retrieval scores of the plurality of first retrieval results; under the condition that the multiple first retrieval results have the first retrieval results with the retrieval scores smaller than the score threshold value, whether the original question is a simple question or not is determined through a large language model; if the original problem is a simple problem, performing query expansion processing on the original problem to obtain a new problem; performing knowledge base retrieval on the new question to obtain a second retrieval result; according to the second retrieval result and the first retrieval result with the retrieval score larger than or equal to the score threshold value in the multiple first retrieval results, answer information is determined.

Description

Technical Field

[0001] This specification relates to the field of artificial intelligence technology, and in particular to a question-answering method, device, electronic device, and medium based on a large language model. Background Art

[0002] RAG (Retrieval Augmented Generation) is a method that combines a very large pre-trained language model with an external retrieval or search mechanism. The core idea of ​​RAG is to allow generative AI models to extract information from a large number of documents during the generation process to generate more accurate and rich text. With the massive increase and complexity of the document corpus, improving the accuracy of RAG has become increasingly challenging. Summary of the invention

[0003] To overcome the problems existing in the related art, this specification provides a question-answering method, device, electronic device and medium based on a large language model.

[0004] According to a first aspect of an embodiment of the present specification, a question-answering method based on a large language model is provided, the method comprising: performing a knowledge base search on an original question to obtain multiple first search results and search scores of the multiple first search results; in a case where there is a first search result whose search score is less than a score threshold among the multiple first search results, using a large language model to determine whether the original question is a simple question; if the original question is a simple question, performing query expansion processing on the original question to obtain a new question; performing a knowledge base search on the new question to obtain a second search result; and determining answer information based on the second search result and the first search result among the multiple first search results whose search score is greater than or equal to the score threshold.

[0005] According to a second aspect of an embodiment of the present specification, a question-answering device based on a large language model is provided, comprising: a first retrieval module, used to perform a knowledge base search on an original question to obtain multiple first retrieval results and retrieval scores of the multiple first retrieval results; a question analysis module, used to use a large language model to determine whether the original question is a simple question if there is a first retrieval result whose retrieval score is less than a score threshold among the multiple first retrieval results; a query expansion module, used to perform query expansion processing on the original question to obtain a new question if the original question is a simple question; a second retrieval module, used to perform a knowledge base search on the new question to obtain a second retrieval result; a first answer determination module, used to determine answer information based on the second retrieval result and the first retrieval result whose retrieval score is greater than or equal to the score threshold among the multiple first retrieval results.

[0006] According to a third aspect of the embodiments of this specification, there is provided an electronic device, including: processor; a memory for storing processor-executable instructions; The processor is configured to execute the question-answering method based on a large language model according to the first aspect or any corresponding embodiment thereof.

[0007] According to a fourth aspect of an embodiment of this specification, a computer-readable storage medium is provided, on which computer instructions are stored, and the computer instructions are used to enable a computer to execute the question-answering method based on a large language model according to the above-mentioned first aspect or any corresponding implementation manner.

[0008] The technical solutions provided by the embodiments of this specification may have the following beneficial effects: In the embodiments of the present specification, by retaining the search results with higher search scores and then performing query expansion or multi-step reasoning on the search results with lower search scores, more accurate search results can be returned, thereby reducing the influence of the search results on the large language model and improving the accuracy of the results returned by the large language model.

[0009] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present specification. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the specification and, together with the description, serve to explain the principles of the specification.

[0011] Figure 1 It is a schematic diagram of a system architecture shown in this specification according to an exemplary embodiment.

[0012] Figure 2 It is a flowchart of a question-answering method based on a large language model according to an exemplary embodiment of this specification.

[0013] Figure 3 It is a flowchart of another question-answering method based on a large language model according to an exemplary embodiment of this specification.

[0014] Figure 4 It is a hardware structure diagram of the computer device where the question-answering device based on the large language model of the embodiment of this specification is located.

[0015] Figure 5 It is a block diagram of a question-answering device based on a large language model according to an exemplary embodiment of this specification. DETAILED DESCRIPTION

[0016] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with this specification. Instead, they are merely examples of devices and methods consistent with some aspects of this specification as detailed in the appended claims.

[0017] The terms used in this specification are for the purpose of describing specific embodiments only and are not intended to limit this specification. The singular forms "a", "an", "the" and "the" used in this specification and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0018] It should be understood that although the terms first, second, third, etc. may be used in this specification to describe various information, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this specification, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0019] Next, the embodiments of this specification are described in detail.

[0020] The following combination Figure 1 The system architecture of the question-answering method and device based on the large language model in the embodiments of this specification is described. It should be noted that: Figure 1 What is shown is merely an example of a system architecture to which the embodiments of this specification can be applied, to help those skilled in the art understand the technical content of this specification, but it does not mean that the embodiments of this specification cannot be used in other devices, systems, environments or scenarios.

[0021] Figure 1 It is a schematic diagram of a system architecture shown in this specification according to an exemplary embodiment.

[0022] like Figure 1 As shown, the system architecture may include, for example, a terminal device, a network and a server. The network is used to provide a medium for a communication link between the terminal device and the server. The network may include various connection types, such as wired and / or wireless communication links, etc.

[0023] Users can use terminal devices to interact with servers through the network to receive or send messages, etc. Various communication client applications can be installed on the terminal devices, such as web browser applications, search applications, instant messaging tools, email clients and / or social platform software, etc.

[0024] The terminal device may be any electronic device having a display screen and supporting web browsing, including but not limited to a smart phone, a tablet computer, a laptop computer, a desktop computer, and the like.

[0025] The server can be a server that provides various services, such as a background management server that provides support for the content browsed by the user using the terminal device. The background management server can analyze and process the received user request and other data, and feed back the processing results (such as web pages, information, or data obtained or generated according to the user request) to the terminal device.

[0026] The following is a detailed description of the question answering method based on a large language model provided in the embodiments of this specification. Figure 2 As shown, Figure 2 This is a flowchart of a question-answering method based on a large language model according to an exemplary embodiment of this specification. The method can be applied to a server, for example. The question-answering method based on a large language model provided in the embodiment of this specification may include the following steps.

[0027] In step 210, a knowledge base search is performed on the original question to obtain a plurality of first search results and search scores of the plurality of first search results.

[0028] According to an embodiment of the present specification, the search score may be used to indicate the degree of association between the first search result and the original question.

[0029] According to an embodiment of the present specification, for example, keywords in the original question can be determined. Based on the keywords, a search is performed in the knowledge base to obtain at least one first recalled document and a recall score of at least one first recalled document. The similarity between each document in the knowledge base and the original question is calculated. Based on the similarity, at least one second recalled document is determined from the knowledge base. The first search result includes a first recalled document and a second recalled document. Then, at least one first recalled document, the recall score of at least one first recalled document, at least one second recalled document, and the similarity of at least one second recalled document are input into a re-ranking model to obtain a retrieval score for each recalled document. The re-ranking model can be used to combine the recall score and the similarity to re-score and rank the recalled documents to obtain a more accurate retrieval score for each recalled document.

[0030] In step 220, if there is a first search result whose search score is less than the score threshold among the multiple first search results, the large language model is used to determine whether the original question is a simple question. According to an embodiment of the present specification, if the search score is less than the score threshold, it means that the search result has a low correlation with the question. The score threshold can be set according to actual needs. Exemplarily, in this embodiment, the average of the search scores of all search results can be calculated as the score threshold.

[0031] According to an embodiment of the present specification, for example, a large language model can be used to analyze whether the original question contains multiple information and whether it is a contrasting question. If the original question does not contain multiple information and the original question is not a contrasting question, the original question is determined to be a simple question, otherwise, the original question is determined to be not a simple question.

[0032] In step 230, if the original question is a simple question, query expansion processing is performed on the original question to obtain a new question.

[0033] According to an embodiment of the present specification, for example, a large language model can be used to determine the domain information of the original question, and a pseudo document related to the original question is generated based on the domain information. The pseudo document is then merged with the original question to obtain a new question.

[0034] For example, the original question may be “How to improve sleep?” The large language model can be used to analyze the original question and generate pseudo documents in related fields, such as “To improve sleep, you need to maintain regular sleep time,” “Avoid caffeine,” and “Use less electronic devices, etc.” The original question and the pseudo documents are then concatenated to obtain a new question.

[0035] In step 240, a knowledge base search is performed on the new question to obtain a second search result.

[0036] According to the embodiments of the present specification, the step of performing knowledge base search on the new question may refer to the step of performing knowledge base search on the original question, and will not be described in detail here.

[0037] In step 250, answer information is determined based on the second search result and a first search result among the plurality of first search results whose search score is greater than or equal to a score threshold.

[0038] According to an embodiment of the present specification, for example, a re-ranking model can be used to adjust the scores of the second search result and the first search result whose search score is greater than or equal to the score threshold among the plurality of first search results to obtain an adjusted score. According to the adjusted score, the k search results (topK) with the highest scores are selected. Wherein, k is a positive integer and can be set according to actual needs. Then, a large language model can be used to generate answer information based on the k search results.

[0039] According to the embodiments of the present specification, by retaining the search results with higher search scores and then performing query expansion or multi-step reasoning on the search results with lower search scores, more accurate search results can be returned, thereby reducing the influence of the search results on the large language model and improving the accuracy of the results returned by the large language model.

[0040] Optionally, if the original question is not a simple question, the large language model is used to decompose the original question to obtain a question chain, which includes multiple sub-questions. The large language model is used to gradually reason about each question in the question chain to obtain a third search result. Then, answer information is determined based on the third search result and the first search result among the multiple first search results whose search score is greater than or equal to the score threshold.

[0041] For example, the original question is: Xiao Ming needs to queue for 3 minutes each time to play bumper cars, and then he can drive the bumper cars for 5 minutes. The amusement park will close in 30 minutes. How many times can Xiao Ming play before it closes? Stage 1: The large language model decomposes the original problem into sub-problems.

[0042] In this embodiment, the large language model can decompose complex problems into smaller sub-problems: in order to answer the question "How many times can Xiao Ming play before it is shut down?", it is necessary to first solve the question "How long does it take to play bumper cars each time?"

[0043] Stage 2: The large language model solves sub-problems one by one and performs chain reasoning.

[0044] In this embodiment, for sub-problem 1, the reasoning process is as follows: Q: How long does it take to play bumper cars each time? A: It takes Xiao Ming 3 minutes to queue up and 5 minutes to drive the bumper car. Therefore, it takes 8 minutes to play bumper cars each time.

[0045] For sub-question 2, the reasoning process is as follows: Q: How many times can Xiao Ming play before closing? A: The amusement park will close in 30 minutes, and each bumper car ride takes 8 minutes. Therefore, Xiao Ming can play 30 ÷ 8 = 3.75 times. Since he cannot drive part of the time, Xiao Ming can play 3 times before closing.

[0046] According to an embodiment of the present specification, for example, the re-ranking model can be used to adjust the scores of the third search result and the first search results whose search scores are greater than or equal to the score threshold among the plurality of first search results to obtain adjusted scores. According to the adjusted scores, the j search results with the highest scores are selected. Wherein, j is a positive integer and can be set according to actual needs. Then, the large language model can be used to generate answer information based on the j search results.

[0047] According to the embodiments of the present specification, by performing query expansion and multi-step reasoning on the question, the retrieval of the document content can be effectively enhanced, so that it can return multiple recalled contents. In addition, after the re-ranking model returns multiple recalled texts with the highest scores, the large language model can better understand the needs of the user's question, so that the large language model can retrieve the correct content in the massive documents.

[0048] According to an embodiment of the present specification, whether the original question is a simple question may be determined based on a routing prompt indicating a large language model.

[0049] Exemplarily, in this embodiment, the route selection prompt may include: Please analyze and determine which category the question belongs to based on the question raised by the user, and output it strictly in the specified format. The judgment rules are as follows: ①If the user's question involves comparison, it is a complex question, otherwise it is a simple question.

[0050] ②If the user's question can be divided into multiple sub-questions, it is a complex question, otherwise it is a simple question.

[0051] ③If it is neither of the above two judgments, it is a simple question.

[0052] For example, the original question is "Now I want to choose a cost-effective router, should I choose brand A or brand B?" The output of the large language model is: complex question According to the embodiments of the present specification, the original question can be decomposed according to the question decomposition prompt to instruct the large language model.

[0053] Exemplarily, in this embodiment, the problem decomposition prompt may include: Please follow the following splitting rules according to the questions raised by the user and output strictly in the specified format.

[0054] The splitting rules are as follows: ① Your purpose is to solve complex problems for users who cannot search the knowledge base well and can return complete information.

[0055] ② Ensure that each split sub-problem is complete and does not generate information irrelevant to the original problem, and the semantics are as close to the original problem as possible.

[0056] ③ Ensure that each sub-problem is a simple problem and no more than 3 sub-problems are generated to meet the needs of chain reasoning.

[0057] For example, the original question is "From Xi'an to Chongqing, is it faster to take the high-speed train or the plane?" Large language model output: Split question 1: How long does it take to take the high-speed rail from Xi'an to Chongqing? Split question 2: How long does it take to fly from Xi'an to Chongqing? According to the embodiments of the present specification, answer information may be generated according to the final inference prompt indicating a large language model.

[0058] Exemplarily, in this embodiment, the final reasoning prompt may include: Please follow the following summary principles based on the questions raised by the user: and output strictly in the specified format.

[0059] The principles are summarized as follows: ① Extract the key points and main concepts related to the topic from the given information.

[0060] ②Based on this information, conduct logical reasoning and draw a comprehensive conclusion about the topic.

[0061] ③ Make sure your answer is simple and clear, and does not contain the semantics of the previous prompt information.

[0062] ④Finally, please present your reasoning summary in a professional and objective tone.

[0063] Example: The user's question is "From Xi'an to Chongqing, is it faster to take the high-speed train or the plane?" topK information: {“It takes about 4 hours to get from Xi'an to Chongqing by high-speed rail”, “It takes about 2 hours to get from Xi'an to Chongqing by plane”} Final reasoning: According to known information, it takes 4 hours to take the high-speed rail from Xi'an to Chongqing, which is more than 2 hours by plane, so it is still faster to take a plane.

[0064] It should be noted that the above prompt case is for reference only, and the prompt can be changed according to specific scenarios.

[0065] like Figure 3 FIG. 1 is a flowchart of another question-answering method based on a large language model according to an exemplary embodiment. The question-answering method based on a large language model may include the following steps: Step 301: The user first asks a question, i.e., the original question. In this embodiment, the text of the question can be segmented and vectorized to obtain a question vector. Vector similarity and keyword search are performed based on the question vector, and a sentence window is used for positioning search and context splicing to obtain text information, i.e., the search result. In this embodiment, ES (Elasticsearch) can be used for keyword search. Elasticsearch is a distributed search and analysis engine.

[0066] Optionally, in this embodiment, 3 text information can be recalled by default. The 3 text information and the recall score of each text information are recalled by calculating the vector similarity and keyword search, and then the retrieval score is returned after the re-ranking model. It is determined whether the retrieval score of the 3 recalled texts is greater than the score threshold. If it is greater, the current recalled text information is retained; otherwise, step 302 is performed. Through this step, some recalled information with low relevance can be eliminated. Exemplarily, in this embodiment, the average calculation function avg() can be used to calculate the average value of the retrieval scores of all retrieval results as the score threshold.

[0067] Step 302: When the search score is less than the score threshold, the current recall text is no longer used. Function calling is designed through LLM and prompt. By optimizing the prompt and limiting the output format, LLM is used to determine whether the current original question is a simple question (a question with only one simple sentence, not involving comparison or multiple sentences). If it is a simple question, proceed to step 303; otherwise, proceed to step 304.

[0068] Step 303: When the question is a simple question, it means that the original question does not retrieve good content, and the original question needs to be expanded.

[0069] Optionally, in this embodiment, the Query2doc method can be used to generate some pseudo documents using LLM, and then these pseudo documents are merged with the original question to form a new question. The pseudo document content needs to be limited to the current search scope, and the generated content is returned as the new recall content.

[0070] Step 304: When the original problem is a complex problem, multi-step reasoning can be performed on the original problem.

[0071] For example, in this embodiment, LLM and prompt can be used to decompose the problem, and then the chain reasoning method can be used to gradually solve the problem, and each question is answered based on the previous question. The prompt clearly states that the answer is based only on the search results, and no other information should be added. Optionally, the LLM answer can also be appropriately integrated with some subjectivity or its understanding of knowledge according to the needs of the scenario.

[0072] Step 305: Finally, the query expansion result or the result after chain reasoning of complex questions and the recalled content with a previous search score greater than or equal to the score threshold are subjected to a re-ranking model, and the score of the knowledge base recall is readjusted according to the real intention of the user's question understanding, and finally the k recalled texts with the highest similarity are returned. In this way, the returned results can be more realistic.

[0073] Step 306: Finally, the k recalled texts are handed over to LLM, and the knowledge base reasoning summary prompt is added. For example, the prompt can be set as: "You are an intelligent customer service. Your goal is to provide accurate information and help the questioner solve the problem as much as possible. Known information: xxx, please answer based on the above information, answer based on the existing knowledge, no guesses are allowed in the answer, and answer in Chinese. If you cannot answer based on the known content, please answer 'The question cannot be answered based on the known information'. The question is xxx". Finally, after LLM reasoning and summary, the answer that the user actually wants is returned.

[0074] Corresponding to the embodiments of the aforementioned method, this specification also provides embodiments of a question-answering device based on a large language model and a terminal to which it is applied.

[0075] The embodiments of the question-answering device based on the large language model in this specification can be applied to computer devices, such as servers or terminal devices. The device embodiments can be implemented by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by the processor in which it is located reading the corresponding computer program instructions in the non-volatile memory into the memory and running them. From the hardware level, if Figure 4 As shown in the figure, it is a hardware structure diagram of the computer device where the question-answering device based on the large language model of the embodiment of this specification is located. Figure 4 In addition to the processor 410, memory 430, network interface 420, and non-volatile memory 440 shown, the server or electronic device where the device 431 is located in the embodiment may also include other hardware according to the actual function of the computer device, which will not be described in detail.

[0076] like Figure 5 As shown, Figure 5 : is a block diagram of a question-answering device based on a large language model according to an exemplary embodiment of the present specification, the device comprising: A first search module 510 is used to search the knowledge base for the original question to obtain a plurality of first search results and search scores of the plurality of first search results; A question analysis module 520, configured to determine whether the original question is a simple question by using a large language model when there is a first search result with a search score less than a score threshold among the plurality of first search results; A query expansion module 530 is used to perform query expansion processing on the original question to obtain a new question if the original question is a simple question; A second search module 540 is used to search the knowledge base for the new question to obtain a second search result; The first answer determination module 550 is used to determine answer information according to the second search result and the first search result whose search score is greater than or equal to the score threshold among the plurality of first search results.

[0077] Optionally, the device may further include: The disassembly module is used to disassemble the original question using a large language model if the original question is not a simple question, and obtain a question chain including multiple sub-questions; A reasoning module, which is used to use a large language model to gradually reason about each question in the question chain to obtain a third retrieval result; The second answer determination module is used to determine answer information according to the third search result and the first search result whose search score is greater than or equal to the score threshold among the plurality of first search results.

[0078] Optionally, the first search result includes a first recalled document and a second recalled document; and the first search module may include: The keyword determination submodule is used to determine the keywords in the original question; A keyword search submodule, used to search the knowledge base according to the keyword to obtain at least one first recalled document and a recall score of at least one first recalled document; The similarity calculation submodule is used to calculate the similarity between each document in the knowledge base and the original question; A similarity retrieval submodule, used to determine at least one second recall document from the knowledge base according to the similarity; The score determination submodule is used to input at least one first recall document, at least one recall score of the first recall document, at least one second recall document and the similarity of at least one second recall document into the re-ranking model to obtain a retrieval score.

[0079] Optionally, the problem analysis module may include: The analysis submodule is used to use the large language model to analyze whether the original question contains multiple aspects of information and whether it is a comparative question; The judgment submodule is used to determine that the original question is a simple question if the original question does not contain multiple information and is not a comparison question; otherwise, the original question is not a simple question.

[0080] Optionally, the query expansion module may include: The domain determination submodule is used to determine the domain information of the original question using the large language model, and generate pseudo documents related to the original question based on the domain information; The merging submodule is used to merge the pseudo document with the original question to obtain a new question.

[0081] According to the embodiments of the present specification, by retaining the search results with higher search scores and then performing query expansion or multi-step reasoning on the search results with lower search scores, more accurate search results can be returned, thereby reducing the influence of the search results on the large language model and improving the accuracy of the results returned by the large language model.

[0082] Correspondingly, the present specification also provides an electronic device, which includes a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to: perform a knowledge base search on the original question to obtain multiple first search results and search scores of the multiple first search results; in the case that there is a first search result with a search score less than a score threshold among the multiple first search results, use a large language model to determine whether the original question is a simple question; if the original question is a simple question, perform query expansion processing on the original question to obtain a new question; perform a knowledge base search on the new question to obtain a second search result; and determine answer information based on the second search result and the first search result among the multiple first search results whose search score is greater than or equal to the score threshold.

[0083] The implementation process of the functions and effects of each module in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, which will not be repeated here.

[0084] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this specification. A person of ordinary skill in the art can understand and implement it without paying creative labor.

[0085] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0086] Those skilled in the art will readily appreciate other embodiments of the specification after considering the specification and practicing the invention claimed herein. The specification is intended to cover any variations, uses or adaptations of the specification that follow the general principles of the specification and include common knowledge or customary techniques in the art that are not claimed in the specification. The specification and examples are to be considered exemplary only, and the true scope and spirit of the specification are indicated by the following claims.

[0087] It should be understood that the present description is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present description is limited only by the appended claims.

[0088] The above description is only a preferred embodiment of this specification and is not intended to limit this specification. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of this specification should be included in the scope of protection of this specification.

Claims

1. A question-answering method based on a large language model, characterized in that: The method comprises: Performing a knowledge base search on the original question to obtain a plurality of first search results and search scores of the plurality of first search results; In the case that there is a first search result whose search score is less than a score threshold among the multiple first search results, using a large language model to determine whether the original question is a simple question; If the original question is a simple question, query expansion processing is performed on the original question to obtain a new question; Performing a knowledge base search on the new question to obtain a second search result; Answer information is determined based on the second search result and a first search result among the plurality of first search results whose search score is greater than or equal to a score threshold.

2. The method according to claim 1, characterized in that The method further comprises: If the original question is not a simple question, the large language model is used to decompose the original question to obtain a question chain, where the question chain includes multiple sub-questions; Using the large language model, gradually reasoning about each question in the question chain to obtain a third retrieval result; Answer information is determined based on the third search result and a first search result among the plurality of first search results whose search score is greater than or equal to a score threshold.

3. The method according to claim 1, characterized in that: The first search result includes a first recalled document and a second recalled document; the knowledge base search for the original question to obtain the first search result and the search score includes: Identify keywords in the original question; Searching the knowledge base according to the keyword to obtain at least one first recalled document and a recall score of the at least one first recalled document; Calculate the similarity between each document in the knowledge base and the original question; Determining at least one second recall document from the knowledge base according to the similarity; The at least one first recalled document, the recall score of the at least one first recalled document, the at least one second recalled document, and the similarity of the at least one second recalled document are input into a re-ranking model to obtain a retrieval score.

4. The method according to claim 1, characterized in that The using of the large language model to determine whether the original question is a simple question includes: Using a large language model to analyze whether the original question contains multiple aspects of information and whether it is a comparative question; If the original question does not contain multi-faceted information and the original question is not a comparison question, then the original question is determined to be a simple question; otherwise, the original question is determined not to be a simple question.

5. The method according to claim 1, characterized in that The query expansion process is performed on the original question to obtain a new question, including: Determine domain information of the original question using a large language model, and generate a pseudo document related to the original question based on the domain information; The pseudo document is merged with the original question to obtain the new question.

6. A question-answering device based on a large language model, characterized in that: The device comprises: A first retrieval module, used to perform a knowledge base search on the original question to obtain a plurality of first retrieval results and retrieval scores of the plurality of first retrieval results; A question analysis module, configured to determine whether the original question is a simple question by using a large language model when there is a first search result whose search score is less than a score threshold among the multiple first search results; A query expansion module, configured to perform query expansion processing on the original question to obtain a new question if the original question is a simple question; A second search module is used to search the knowledge base for the new question to obtain a second search result; The first answer determination module is used to determine answer information according to the second search result and a first search result among the plurality of first search results whose search score is greater than or equal to a score threshold.

7. The device according to claim 6, characterized in that The device also includes: A disassembly module, used for disassembling the original question using a large language model to obtain a question chain if the original question is not a simple question, wherein the question chain includes a plurality of sub-questions; A reasoning module, used for using a large language model to gradually reason about each question in the question chain to obtain a third search result; The second answer determination module is used to determine answer information according to the third search result and the first search result whose search score is greater than or equal to the score threshold among the multiple first search results.

8. The device according to claim 6, characterized in that The first search result includes a first recalled document and a second recalled document; and the first search module includes: A keyword determination submodule, used to determine the keywords in the original question; A keyword search submodule, used to search the knowledge base according to the keyword to obtain at least one first recalled document and a recall score of the at least one first recalled document; A similarity calculation submodule, used to calculate the similarity between each document in the knowledge base and the original question; A similarity retrieval submodule, configured to determine at least one second recalled document from the knowledge base according to the similarity; The score determination submodule is used to input the at least one first recalled document, the recall score of the at least one first recalled document, the at least one second recalled document and the similarity of the at least one second recalled document into the re-ranking model to obtain a retrieval score.

9. The device according to claim 6, characterized in that The problem analysis module includes: An analysis submodule, used to use a large language model to analyze whether the original question contains multiple aspects of information and whether it is a comparative question; The judgment submodule is used to determine that the original question is a simple question if the original question does not contain multi-faceted information and the original question is not a comparison question; otherwise, determine that the original question is not a simple question.

10. The device according to claim 6, characterized in that The query expansion module comprises: A domain determination submodule, configured to determine domain information of the original question using a large language model, and generate a pseudo document related to the original question based on the domain information; The merging submodule is used to merge the pseudo document with the original question to obtain the new question.

11. An electronic device, comprising: processor; a memory for storing processor-executable instructions; The processor is configured to execute the method according to any one of claims 1 to 5.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the method according to any one of claims 1 to 5.

Images