Question and answer method and device and computer equipment

Abstract

The invention discloses a question and answer method and device and computer equipment. The method comprises the following steps: receiving a target question; the big language model is adopted to call internally stored knowledge to analyze the target question, an answer and a reason corresponding to the target question are obtained, and the reason is used for representing thinking logic of the big language model in the answer generation process; a pre-trained retrieval decision model is adopted to analyze the answer and reason corresponding to the target question, a target decision is obtained, and the target decision is used for representing whether retrieval needs to be conducted from an external knowledge base based on the target question or not; and under the condition that the target decision indication needs to be retrieved from the external knowledge base based on the target question, updating an answer and a reason corresponding to the target question according to information retrieved from the external knowledge base to obtain a final answer and a final reason.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and more specifically, to a question-answering method, apparatus, and computer device. Background Art

[0002] Due to the rapid development of artificial intelligence, an artificial intelligence framework RAG (Retrieval-Augmented Generation) has been proposed, which combines the advantages of traditional information retrieval systems (such as databases) with the functions of LLM (Large Language Model). By combining this additional knowledge with its own language skills, LLM can write more accurate, timely and specific texts. In LLM-specific fields or knowledge-intensive tasks, especially when dealing with queries that exceed its training data or require current information, "hallucinations" often occur. RAG is one of the most effective ways to solve hallucinations. Insufficiency of existing RAG technology: RAG technology usually involves introducing external knowledge to enhance the output of LLM (large language model) during the generation process. However, when to retrieve and how to use the retrieved knowledge is a key issue. The current RAG system lacks flexible retrieval strategies, the model lacks awareness of the scope of its own knowledge, and the smaller the parameters of the large model, the less it knows what it knows, and cannot dynamically adjust the content and timing of the retrieval according to the needs of the current generation. The existing RAG system may lack an effective conflict detection and resolution mechanism. When external knowledge conflicts with internal knowledge, the system needs to be able to accurately identify the conflict and adopt appropriate resolution strategies. The more external knowledge there is, the greater the possibility of conflict. When answering complex questions, it may be necessary to combine multiple internal and external knowledge. However, current RAG systems may lack efficient knowledge fusion and trade-off mechanisms. Summary of the invention

[0003] The embodiments of the present application provide a question-answering method, apparatus, and computer device to at least solve the technical problem in the related art that the question-answering accuracy of the model is low due to frequent retrieval of additional knowledge.

[0004] According to one aspect of an embodiment of the present application, a question-answering method is provided, comprising: receiving a target question; using a large language model to call internally stored knowledge to analyze the target question, and obtaining an answer and reason corresponding to the target question, wherein the reason is used to represent the thinking logic of the large language model in the process of generating the answer; using a pre-trained retrieval decision model to analyze the answer and reason corresponding to the target question, and obtaining a target decision, wherein the target decision is used to characterize whether it is necessary to retrieve from an external knowledge base based on the target question; when the target decision indicates that it is necessary to retrieve from an external knowledge base based on the target question, updating the answer and reason corresponding to the target question according to the information retrieved from the external knowledge base, and obtaining a final answer and reason.

[0005] Optionally, a pre-trained retrieval decision model is used to analyze the answers and reasons corresponding to the target question to obtain a target decision, including: processing the hidden state representation of the answers and reasons of the target question to obtain a final hidden state representation; using the pre-trained retrieval decision model to calculate the logical value of the final hidden state representation; when the logical value is higher than a preset threshold, determining that the target decision does not need to be retrieved from an external knowledge base based on the target question; when the logical value is lower than a preset threshold, determining that the target decision needs to be retrieved from an external knowledge base based on the target question.

[0006] Optionally, the hidden state representations of the answer and reasons for the target question are processed to obtain a final hidden state representation, including: performing mean calculation on the hidden state representations of the answer and reasons for the target question on the label dimension to obtain a mean calculation result; and normalizing the mean calculation result to obtain the final hidden state representation.

[0007] Optionally, the pre-trained retrieval decision model is trained in the following manner, including: obtaining a training data set, the training data set including multiple groups of data, wherein each group of data includes: a question, a generated reason, a generated answer and a label, wherein the label is used to indicate whether a retrieval based on the question is required; using a cross entropy loss function and the training data set to train an initial model to obtain the pre-trained retrieval decision model.

[0008] Optionally, obtaining a training data set includes: obtaining a plurality of questions; generating two sets of reasons and answers for each question, wherein a first set of reasons and answers are generated by the large language model, and a second set of reasons and answers are obtained by adjusting the first set of reasons and answers using retrieved knowledge; determining the first set of reasons and answers corresponding to each question and a label of each question as one set of data in the training data set; and determining the second set of reasons and answers corresponding to each question and the label of each question as another set of data in the training data set.

[0009] Optionally, the answer and reason corresponding to the target question are updated according to the information retrieved from the external knowledge base to obtain the final answer and reason, including: obtaining the updated answer and reason, and obtaining the final hidden state representation corresponding to the updated answer and reason; determining the final hidden state representation corresponding to the updated answer and reason as the target hidden state representation; repeatedly using the pre-trained retrieval decision model to analyze the target hidden state representation until the target hidden state representation no longer needs to be retrieved or the maximum number of retrievals is reached, to obtain the final answer and reason.

[0010] Optionally, the method further includes: when the target decision indicates that there is no need to retrieve from an external knowledge base based on the target question, determining the answer and reason corresponding to the target question as the final answer and reason.

[0011] According to another aspect of an embodiment of the present application, a question-and-answer device is also provided, including: a receiving module for receiving a target question; a first analysis module for using a large language model to call internally stored knowledge to analyze the target question, and obtain an answer and reason corresponding to the target question, and the reason is used to represent the thinking logic of the large language model in the process of generating the answer; a second analysis module for using a pre-trained retrieval decision model to analyze the answer and reason corresponding to the target question, and obtain a target decision, and the target decision is used to characterize whether it is necessary to retrieve from an external knowledge base based on the target question; a reply module for updating the answer and reason corresponding to the target question according to the information retrieved from the external knowledge base to obtain a final answer and reason when the target decision indicates that it is necessary to retrieve from an external knowledge base based on the target question.

[0012] According to another aspect of the embodiment of the present application, a computer device is also provided, including: a memory and a processor, wherein the memory is used to store program instructions; the processor is connected to the memory and is used to execute the above-mentioned question-and-answer method.

[0013] According to another aspect of the embodiments of the present application, a non-volatile storage medium is provided, which includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the above-mentioned question-and-answer method by running the computer program.

[0014] According to another aspect of the embodiments of the present application, a computer program product is provided, including computer instructions, which implement the above-mentioned question-and-answer method when executed by a processor.

[0015] In an embodiment of the present application, a target question is received; a large language model is used to call internally stored knowledge to analyze the target question, and an answer and reason corresponding to the target question are obtained, wherein the reason is used to represent the thinking logic of the large language model in the process of generating the answer; a pre-trained retrieval decision model is used to analyze the answer and reason corresponding to the target question, and a target decision is obtained, wherein the target decision is used to characterize whether it is necessary to retrieve from an external knowledge base based on the target question; when the target decision indicates that it is necessary to retrieve from an external knowledge base based on the target question, the answer and reason corresponding to the target question are updated according to the information retrieved from the external knowledge base to obtain a final answer and reason, thereby achieving the purpose of using a retrieval decision model to analyze the answer to obtain a result of whether to retrieve, thereby avoiding unnecessary additional retrieval, thereby achieving the technical effect of avoiding the low accuracy of the generated answer due to the conflict between the additional retrieval knowledge and the knowledge stored internally in the model, thereby solving the technical problem in the related art that the model's question and answer accuracy is low due to frequent retrieval of additional knowledge. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0017] Figure 1 is a hardware structure block diagram of a computer terminal for implementing a question-answering method according to an embodiment of the present application;

[0018] Figure 2 is a flow chart of a question-and-answer method according to an embodiment of the present application;

[0019] Figure 3 is a flow chart of input and output of a retrieval decision model according to an embodiment of the present application;

[0020] Figure 4 is a flow chart of RAG system input and output according to an embodiment of the present application;

[0021] Figure 5It is a structural diagram of a question-and-answer device according to an embodiment of the present application. DETAILED DESCRIPTION

[0022] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0023] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0024] The information collected in the embodiments of the present application is information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with the relevant laws, regulations and standards of the relevant regions, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or reject automated decision-making results; if the user chooses to reject, the expert decision-making process will be entered.

[0025] In order to solve the problems existing in the related art, the embodiment of the present application provides a question-answering method, which can be run on Figure 1 In the computer terminal shown, the computer terminal is explained below.

[0026] The question-and-answer method embodiment provided in the embodiment of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 FIG. 1 shows a hardware structure block diagram of a computer terminal for implementing the question-answering method. Figure 1As shown, the computer terminal 10 may include one or more (102a, 102b, ..., 102n are used to illustrate) processors (the processor may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission module 106 for communication functions connected via a wired and / or wireless network. In addition, it may also include: a display, a keyboard, a cursor control device, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, and a BUS bus. Those skilled in the art can understand that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0027] It should be noted that the one or more processors and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 10. As described in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0028] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the question-and-answer method in the embodiment of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, realizing the above-mentioned question-and-answer method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0029] The transmission module 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission module 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission module 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0030] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 .

[0031] It should be noted that, in some optional embodiments, the above Figure 1 The computer terminal shown may include hardware elements (including circuits), software elements (including computer code stored on a computer-readable medium), or a combination of hardware elements and software elements. It should be noted that Figure 1 This is merely one example of a particular embodiment and is intended to illustrate the types of components that may be present in the computer terminal described above.

[0032] In the above-mentioned operating environment, an embodiment of the present application provides an embodiment of a question-and-answer method. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0033] Figure 2 is a flow chart of a question-answering method according to an embodiment of the present application, such as Figure 2 As shown, the method comprises the following steps:

[0034] Step S202, receiving a target question;

[0035] Step S204, using the large language model to call the internally stored knowledge to analyze the target question, and obtain the answer and reason corresponding to the target question, wherein the reason is used to represent the thinking logic of the large language model in the process of generating the answer;

[0036] In step S204, the reason for the question represents the internal thinking process or knowledge clues that the large language model (LLM) relies on when generating an answer. When the LLM attempts to answer a question raised by a user, it will not only give a direct answer, but also generate a reason to explain or illustrate why the answer is reasonable or correct. The reason includes the logical reasoning or association between concepts performed by the LLM when answering the question. For example, if the question is about the judgment of a legal case, the reason may include explaining the legal provisions or precedents on which the judgment is based.

[0037] Step S206, using a pre-trained retrieval decision model to analyze the answer and reason corresponding to the target question to obtain a target decision, wherein the target decision is used to indicate whether it is necessary to search from an external knowledge base based on the target question;

[0038] Step S208, when the target decision indicates that a search is required from an external knowledge base based on the target question, the answer and reason corresponding to the target question are updated according to the information retrieved from the external knowledge base to obtain a final answer and reason.

[0039] Through the above steps S202 to S208, by receiving the target question; using the large language model to call the internally stored knowledge to analyze the target question, obtaining the answer and reason corresponding to the target question, the reason is used to represent the thinking logic of the large language model in the process of generating the answer; using the pre-trained retrieval decision model to analyze the answer and reason corresponding to the target question, obtaining the target decision, the target decision is used to characterize whether it is necessary to retrieve from the external knowledge base based on the target question; when the target decision indicates that it is necessary to retrieve from the external knowledge base based on the target question, the answer and reason corresponding to the target question are updated according to the information retrieved from the external knowledge base to obtain the final answer and reason, thereby achieving the purpose of using the retrieval decision model to analyze the answer to obtain the result of whether to retrieve, thereby avoiding unnecessary additional retrieval, thereby achieving the technical effect of avoiding the low accuracy of the generated answer due to the conflict between the additional retrieval knowledge and the knowledge stored in the model, thereby solving the technical problem in the related art that the model's question-answering accuracy is low due to frequent retrieval of additional knowledge. The following is a detailed description.

[0040] It should also be noted that the question-answering method provided in this application achieves the following effects by analyzing the answers and reasons corresponding to the target question before the retrieval to determine whether additional retrieval is needed, such as: reducing the complexity of the query: in large model end-to-end applications, some queries may require multiple retrieval steps, while some queries do not require additional retrieval at all. Traditional retrieval and generation processes may not be optimized for these diverse needs. Reduce the conflict between internal knowledge and external knowledge: in the RAG system, the conflict between external knowledge and the internal knowledge of the model may lead to inconsistent output, and the question-answering method proposed in this application reduces the probability of knowledge conflict by reducing unnecessary retrieval steps; improve the reliability and accuracy of the model: the question-answering method provided in this application retrieves external documents and knowledge only when necessary, reducing the interference of invalid information on the final generation results of the model, thereby improving the accuracy and reliability of the model in open domain question-answering tasks; improve computational efficiency: by reducing unnecessary retrieval steps, processing efficiency is improved.

[0041] In some embodiments of the present application, a pre-trained retrieval decision model is used to analyze the answer and reason corresponding to the target question, and the specific steps for obtaining the target decision are as follows: the hidden state representation of the answer and reason of the target question is processed to obtain a final hidden state representation; the pre-trained retrieval decision model is used to calculate the logical value of the final hidden state representation; when the logical value is higher than a preset threshold, it is determined that the target decision does not need to be retrieved from an external knowledge base based on the target question; when the logical value is lower than the preset threshold, it is determined that the target decision needs to be retrieved from an external knowledge base based on the target question.

[0042] It is understandable that the external knowledge base is in contrast to the knowledge stored inside the model and does not belong to the knowledge stored in this application of the model.

[0043] It is also important to note that the retrieval decision model is a feed-forward neural network with one hidden layer and one output layer for binary classification. The retrieval decision model uses the final hidden state representation of the rationale and answer generated by the large language model as input and outputs whether additional retrieval is needed.

[0044] In some embodiments of the present application, the hidden state representations of the answer and reasons for the target question are processed to obtain the following specific steps for obtaining the final hidden state representation: performing mean calculation on the hidden state representations of the answer and reasons for the target question on the label dimension to obtain the mean calculation result; and normalizing the mean calculation result to obtain the final hidden state representation.

[0045] The specific conversion process is shown as follows:

[0046]

[0047] In the formula, represents the final hidden state representation of the answer and reason output at the kth layer of the model, Norm represents the normalization operation, Mean represents the average value calculation, l represents the layer of the model, dim=0 represents the label dimension, Indicates that it has d model dimensional real number space.

[0048] In some embodiments of the present application, the specific process of training the pre-trained retrieval decision model is as follows: obtaining a training data set, the training data set including multiple groups of data, wherein each group of data includes: a question, a generated reason, a generated answer and a label, wherein the label is used to indicate whether a retrieval based on the question is required; using a cross entropy loss function and the training data set to train the initial model to obtain the pre-trained retrieval decision model.

[0049] The cross entropy loss function is shown as follows:

[0050]

[0051] In the formula, L represents the loss function value, N represents the number of samples in the training data set, and y i represents the label of the i-th sample, p i Represents the predicted probability of the i-th sample, and represents the probability that needs to be retrieved.

[0052] For example, each set of data includes the input represented by the final hidden state and a label indicating whether additional retrieval is required.

[0053] Wherein, obtaining a training data set includes: obtaining a plurality of questions; generating two sets of reasons and answers for each question respectively, wherein a first set of reasons and answers are generated by the large language model, and a second set of reasons and answers are obtained by adjusting the first set of reasons and answers using retrieved knowledge; determining the first set of reasons and answers corresponding to each question and a label of each question as a set of data in the training data set; and determining the second set of reasons and answers corresponding to each question and a label of each question as another set of data in the training data set.

[0054] For each question (q), two versions of r (reason) and a^ (answer) are generated: one without retrieval and the other with retrieval. This dual approach helps the model learn when retrieval improves performance. By comparing the hidden state representations in retrieval and non-retrieval cases, the model can better detect cases where retrieval may improve accuracy. The training dataset includes four types of examples: (with retrieval, y=0), (with retrieval, y=1), (without retrieval, y=0), and (without retrieval, y=1). The model is trained using the generated dataset and cross-entropy loss.

[0055] It can be understood that y=0 indicates that an additional search is required for the question, and y=1 indicates that an additional search is not required for the question.

[0056] like Figure 3 As shown in the figure, the input and output process of the retrieval decision model is as follows: the large language model receives the question set QADataset, and generates four types of answers based on the LLM, for example: Wrong w / o Retireval (no retrieval, wrong answer), Correct w / o Retireval (no retrieval, correct answer), Wrong with Retireval (retrieval has been performed, wrong answer), Correct with Retireval (retrieval has been performed, correct answer), and converts the above four types of answers into hidden state representations (Hidden states), and inputs the hidden state representation and label (Label) into the retrieval decision model to obtain the decision result, which is used to indicate whether external knowledge retrieval is needed for the question.

[0057] In some embodiments of the present application, the answer and reason corresponding to the target question are updated according to the information retrieved from the external knowledge base to obtain the final answer and reason, including: obtaining the updated answer and reason, and obtaining the final hidden state representation corresponding to the updated answer and reason; determining the final hidden state representation corresponding to the updated answer and reason as the target hidden state representation; repeatedly using the pre-trained retrieval decision model to analyze the target hidden state representation until the target hidden state representation no longer needs to be retrieved or the maximum number of retrievals is reached, to obtain the final answer and reason.

[0058] Specifically, if the retrieval decision model determines that a search is needed, the system will enter an iterative process. In each iteration, the LLM will process the updated information (including the retrieved external information) again to generate new reasons and answers. The retrieval decision model will again evaluate whether the new reasons and answers need further search until no search is needed or the preset maximum number of iterations is reached.

[0059] In a case where the target decision indication does not need to be retrieved from an external knowledge base based on the target question, the answer and reason corresponding to the target question are determined as the final answer and reason.

[0060] like Figure 4 As shown in the figure, the large language model receives the target question (Q), passes through the attention layer (Attention Layer) and the fully connected feedforward neural network layer (FNN Layer) in sequence, and inputs the final hidden state representation corresponding to the generated answer and reason into the retrieval decision model. The retrieval decision model decides whether to retrieve (Prober decides wether to Retrieve or Not). If retrieval is required, after retrieval, new answers and reasons (A) are formed based on the retrieved knowledge and the knowledge stored in the model itself.

[0061] In order to test the model performance of the question-answering method proposed in this application, the embodiments of this application respectively test the accuracy and exact match rate of the answer results under different question-answering models and whether or not a search is performed. It can be seen that the accuracy of a single search in all cases is higher than the result without a search. Compared with a single search and no search, the accuracy of the question-answering method provided by the embodiments of this application is improved. The embodiments of this application also test the accuracy of the answers of different question-answering models when using the same data set. The answer generated by the method provided by the embodiments of this application has the highest accuracy.

[0062] Figure 5 A question-answering device according to an embodiment of the present application includes:

[0063] A receiving module 50, used for receiving a target question;

[0064] A first analysis module 52 is used to analyze the target question by using the large language model to call the internally stored knowledge to obtain the answer and reason corresponding to the target question, where the reason is used to represent the thinking logic of the large language model in the process of generating the answer;

[0065] A second analysis module 54 is used to analyze the answer and reason corresponding to the target question using a pre-trained retrieval decision model to obtain a target decision, wherein the target decision is used to indicate whether it is necessary to search from an external knowledge base based on the target question;

[0066] The reply module 56 is used to update the answer and reason corresponding to the target question according to the information retrieved from the external knowledge base to obtain the final answer and reason when the target decision indication needs to be retrieved from the external knowledge base based on the target question.

[0067] The above-mentioned question-and-answer device receives a target question; uses a large language model to call internally stored knowledge to analyze the target question, and obtains an answer and reason corresponding to the target question, and the reason is used to represent the thinking logic of the large language model in the process of generating the answer; uses a pre-trained retrieval decision model to analyze the answer and reason corresponding to the target question to obtain a target decision, and the target decision is used to characterize whether it is necessary to retrieve from an external knowledge base based on the target question; when the target decision indicates that it is necessary to retrieve from an external knowledge base based on the target question, the answer and reason corresponding to the target question are updated according to the information retrieved from the external knowledge base to obtain a final answer and reason, thereby achieving the purpose of using a retrieval decision model to analyze the answer to obtain a result of whether to retrieve, thereby avoiding unnecessary additional retrieval, thereby achieving the technical effect of avoiding the low accuracy of the generated answer due to the conflict between the additional retrieval knowledge and the knowledge stored in the model, thereby solving the technical problem in the related art that the model's question-and-answer accuracy is low due to frequent retrieval of additional knowledge.

[0068] The second analysis module 54 includes: an analysis submodule, which is used to use a pre-trained retrieval decision model to analyze the answer and reason corresponding to the target question to obtain a target decision, including: processing the hidden state representation of the answer and reason of the target question to obtain a final hidden state representation; using the pre-trained retrieval decision model to calculate the logical value of the final hidden state representation; when the logical value is higher than a preset threshold, determining that the target decision does not need to be retrieved from an external knowledge base based on the target question; when the logical value is lower than the preset threshold, determining that the target decision needs to be retrieved from an external knowledge base based on the target question.

[0069] The analysis submodule includes: a processing unit, which is used to process the hidden state representation of the answer and reason of the target question to obtain a final hidden state representation, including: performing mean calculation on the hidden state representation of the answer and reason of the target question on the label dimension to obtain a mean calculation result; normalizing the mean calculation result to obtain the final hidden state representation.

[0070] The second analysis module 54 also includes: a training submodule, used to obtain a training data set, wherein the training data set includes multiple groups of data, wherein each group of data includes: a question, a generated reason, a generated answer and a label, wherein the label is used to indicate whether a retrieval based on the question is required; using a cross entropy loss function and the training data set to train the initial model to obtain the pre-trained retrieval decision model.

[0071] The training submodule includes: an acquisition unit, which is used to acquire a training data set, including: acquiring multiple questions; generating two sets of reasons and answers for each question, wherein the first set of reasons and answers are generated by the large language model, and the second set of reasons and answers are obtained by adjusting the first set of reasons and answers using the retrieved knowledge; determining the first set of reasons and answers corresponding to each question and the label of each question as one set of data in the training data set; determining the second set of reasons and answers corresponding to each question and the label of each question as another set of data in the training data set.

[0072] The reply module 56 includes: a reply submodule, which is used to update the answer and reason corresponding to the target question according to the information retrieved from the external knowledge base to obtain the final answer and reason, including: obtaining the updated answer and reason, and obtaining the final hidden state representation corresponding to the updated answer and reason; determining the final hidden state representation corresponding to the updated answer and reason as the target hidden state representation; repeatedly using the pre-trained retrieval decision model to analyze the target hidden state representation until the target hidden state representation no longer needs to be retrieved or the maximum number of retrievals is reached to obtain the final answer and reason.

[0073] The reply submodule includes a reply unit, which is used to determine the answer and reason corresponding to the target question as the final answer and reason when the target decision indication does not need to be retrieved from the external knowledge base based on the target question.

[0074] It should be noted that Figure 5 The question-answering device shown is used to perform Figure 2 The question-and-answer method shown in the figure, therefore the relevant explanations in the above question-and-answer method are also applicable to the question-and-answer device, and will not be repeated here.

[0075] An embodiment of the present application also provides a computer device, including: a memory and a processor, wherein the memory is used to store program instructions; the processor is connected to the memory and is used to execute the above-mentioned question-and-answer method.

[0076] An embodiment of the present application also provides a non-volatile storage medium, which includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the above-mentioned question-and-answer method by running the computer program.

[0077] An embodiment of the present application also provides a computer program product, including computer instructions, which, when executed by a processor, implement the steps of the question-and-answer method in the present application.

[0078] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0079] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0080] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0081] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0082] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0083] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0084] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A question-answering method, characterized in that: include: Receive target questions; The large language model is used to call the internally stored knowledge to analyze the target question, and obtain the answer and reason corresponding to the target question, wherein the reason is used to represent the thinking logic of the large language model in the process of generating the answer; Using a pre-trained retrieval decision model to analyze the answer and reason corresponding to the target question to obtain a target decision, wherein the target decision is used to indicate whether it is necessary to search from an external knowledge base based on the target question; In the case where the target decision indication needs to be retrieved from an external knowledge base based on the target question, the answer and reason corresponding to the target question are updated according to the information retrieved from the external knowledge base to obtain a final answer and reason.

2. The method according to claim 1, characterized in that The pre-trained retrieval decision model is used to analyze the answers and reasons corresponding to the target question to obtain the target decision, including: Processing the hidden state representations of the answer and the reason for the target question to obtain a final hidden state representation; Calculating the logical value represented by the final hidden state using the pre-trained retrieval decision model; When the logic value is higher than a preset threshold, determining that the target decision does not need to be retrieved from an external knowledge base based on the target question; When the logic value is lower than a preset threshold, it is determined that the target decision needs to be retrieved from an external knowledge base based on the target question.

3. The method according to claim 2, characterized in that The hidden state representations of the answer and the reason for the target question are processed to obtain a final hidden state representation, including: Performing mean calculation on the hidden state representations of the answer and reason of the target question on the label dimension to obtain a mean calculation result; The mean calculation result is normalized to obtain the final hidden state representation.

4. The method according to claim 1, characterized in that: The pre-trained retrieval decision model is trained in the following ways, including: Obtaining a training data set, wherein the training data set includes multiple sets of data, wherein each set of data includes: a question, a generated reason, a generated answer, and a label, wherein the label is used to indicate whether a search based on the question is required; The initial model is trained using a cross entropy loss function and the training data set to obtain the pre-trained retrieval decision model.

5. The method according to claim 4, characterized in that Get the training dataset, including: Get multiple questions; Generate two sets of reasons and answers for each question, wherein the first set of reasons and answers are generated by the large language model, and the second set of reasons and answers are obtained by adjusting the first set of reasons and answers using the retrieved knowledge; Determine the first set of reasons and answers corresponding to each question and the label of each question as a set of data in the training data set; A second set of reasons and answers corresponding to each question and a label of each question are determined as another set of data in the training data set.

6. The method according to claim 1, characterized in that The answer and reason corresponding to the target question are updated according to the information retrieved from the external knowledge base to obtain a final answer and reason, including: Obtaining updated answers and reasons, and obtaining final hidden state representations corresponding to the updated answers and reasons; Determine the final hidden state representation corresponding to the updated answer and reason as the target hidden state representation; The pre-trained retrieval decision model is repeatedly used to analyze the target hidden state representation until the target hidden state representation no longer needs to be retrieved or the maximum number of retrievals is reached, thereby obtaining the final answer and reason.

7. The method according to claim 6, characterized in that The method further comprises: In a case where the target decision indication does not need to be retrieved from an external knowledge base based on the target question, the answer and reason corresponding to the target question are determined as the final answer and reason.

8. A question-answering device, characterized in that: include: A receiving module, used for receiving a target question; A first analysis module is used to use the large language model to call the internally stored knowledge to analyze the target question, and obtain the answer and reason corresponding to the target question, wherein the reason is used to represent the thinking logic of the large language model in the process of generating the answer; A second analysis module is used to analyze the answer and reason corresponding to the target question using a pre-trained retrieval decision model to obtain a target decision, wherein the target decision is used to indicate whether it is necessary to search from an external knowledge base based on the target question; The reply module is used to update the answer and reason corresponding to the target question according to the information retrieved from the external knowledge base to obtain the final answer and reason when the target decision indicates that a retrieval is required from the external knowledge base based on the target question.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory is used to store program instructions; The processor is connected to the memory, and is used to execute the question-answering method described in any one of claims 1 to 7.

10. A computer program product comprising computer instructions, characterized in that: When the computer instructions are executed by a processor, the question-answering method described in any one of claims 1 to 7 is implemented.

Images