Question and answer method and computing device

Abstract

The embodiment of the invention provides a question and answer method and computing equipment, and relates to the technical field of computers, in the multi-round retrieval process, information provided by a catalog of a document is fully utilized, the retrieval accuracy is remarkably improved, and the large language model reasoning effect is improved. The retrieval result of each round of retrieval comprises at least one target text block until the plurality of retrieved target text blocks meet a target condition; based on the multiple target text blocks, obtaining answers of the user questions; wherein the knowledge base comprises a plurality of text blocks, the text blocks are associated with respective source information, and the source information is used for indicating documents to which the text blocks belong or positions of the text blocks in the documents to which the text blocks belong; retrieval conditions of the first round of retrieval comprise user questions; the retrieval condition of at least one round of retrieval process in the rest of each round of retrieval comprises source information associated with at least one retrieved target text block.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of computer, and particularly relate to a question answering method and a computing device. BACKGROUND

[0002] Large language models (LLM) are currently commonly deployed in intelligent question answering systems for reasoning the answers to user input questions. However, the knowledge of large language models is often from pre-training data, which is usually not dynamically updated, and hallucination problems are prone to occur when reasoning answers, misleading users.

[0003] Therefore, the industry proposes retrieval-augmented generation (RAG) technology, which uses an external knowledge base to provide large language models with more abundant knowledge to improve reasoning effects.

[0004] However, in the current intelligent question answering system using RAG, the retrieval process has defects, which easily leads to imperfect retrieval results and low matching degree with the question, affecting the reasoning effect of the large language model. SUMMARY

[0005] Embodiments of the present application provide a question answering method and a computing device, which fully utilize the source information of the text blocks through multi-round retrieval, and accurately obtain the context of the text blocks or other highly relevant text blocks according to the documents to which the text blocks belong or the positions of the text blocks in the documents indicated by the source information, thereby significantly improving the accuracy of retrieval and the perfection degree of retrieval results.

[0006] In a first aspect, embodiments of the present application provide a question answering method, which includes: based on a user question, performing multi-round retrieval on a knowledge base, the retrieval results of each round of retrieval including at least one target text block, until the plurality of target text blocks retrieved meet a target condition; based on the plurality of target text blocks, obtaining an answer to the user question; wherein the knowledge base includes a plurality of text blocks, and the text blocks are associated with respective source information, the source information being used to indicate the documents to which the text blocks belong or the positions of the text blocks in the documents; the retrieval condition of the first round of retrieval includes the user question; and the retrieval condition of at least one of the remaining rounds of retrieval includes the source information associated with at least one target text block that has been retrieved.

[0007] It can be understood that through multiple rounds of retrieval, the completeness and accuracy of the retrieval results can be gradually improved. Moreover, in the retrieval process, by using the source information associated with the retrieved target text block, the context originally in the same document or the highly associated text block in different documents can be accurately obtained, further improving the completeness of the retrieval results and helping to improve the reasoning effect of the model.

[0008] In a possible implementation, the source information of the first text block is used to indicate that the position of the first text block in the first document is under the first directory node, and the first text block is any target text block that has been retrieved; and in at least one round of retrieval in the remaining rounds of retrieval, the source information of the first text block is used as a retrieval condition to obtain a third text block associated with the source information of the first text block in the knowledge base, and the third text block is a text block under the first directory node of the first document and other than the first text block.

[0009] It can be understood that the retrieval based on the similarity between text blocks can generally find similar text blocks, but such retrieval results are usually not perfect, for example, only a certain step highly related to the question is retrieved, but the complete operation process includes multiple steps. However, by using the source information of the target text block, the target text block can be accurately positioned in the first directory node of the first document, and the text blocks under the first directory node and other than the first text block can be obtained, thereby significantly improving the completeness of the retrieval results and helping to improve the reasoning effect of the model.

[0010] In a possible implementation, the source information of the first text block is used to indicate that the first text block belongs to the first document; the first text block is any target text block that has been retrieved; and in at least one round of retrieval in the remaining rounds of retrieval, the user question and the directory information of the first document are input into the large language model to obtain at least one second directory node output by the large language model; and the second directory node is used as a retrieval condition to obtain a third text block associated with the second directory node of the first document in the knowledge base.

[0011] It can be understood that by using the reasoning capability of the large language model, it can be determined which directory nodes in the same document need to be retrieved based on the current retrieval result (the first text block), and then the text blocks under the second directory node are obtained, thereby improving the completeness of the retrieval results and helping to improve the reasoning effect of the model.

[0012] In a possible implementation, the source information of the first text block is used to indicate that the first text block belongs to the first document; the first text block is any target text block that has been retrieved; and each round of the remaining at least one round of retrieval includes: taking the source information of the first text block as a retrieval condition, and based on a pre-generated graph relationship, obtaining a fourth text block in the knowledge base that is associated with the source information of the first text block, where the graph relationship is used to indicate an association relationship between different documents in the knowledge base, and the fourth text block is a text block in a second document that is associated with the first document in the knowledge base.

[0013] It can be understood that, by using the source information of the first text block (such as the document name to which the first text block belongs) and the graph relationship that indicates the association relationship between different documents, the text blocks in other documents that are associated with the first document to which the first text block belongs can be accurately obtained, and the completeness of the retrieval result is further improved.

[0014] In a possible implementation, after each round of retrieval ends, at least the retrieval result obtained in the last time is input into the large language model to obtain a retrieval condition for a next round of retrieval process output by the large language model, and the retrieval condition for the next round of retrieval process includes source information associated with at least one target text block that has been retrieved.

[0015] It can be understood that, by using the reasoning capability of the large language model to give the retrieval condition for the next round of retrieval process, and because the target text blocks in the retrieval result have associated source information, the large language model can give a more targeted and specific retrieval condition based on the source information of the target text blocks in the reasoning process, for example, “retrieve the text under the directory node B of the document A”, instead of only giving a simple keyword for global retrieval, which significantly improves the efficiency and accuracy of retrieval.

[0016] In a possible implementation, the answer to the user question is obtained based on the plurality of target text blocks, including: obtaining a target input sequence based on the user question and the plurality of target text blocks; inputting the target input sequence into the large language model to obtain an answer output by the large language model; and the large language model is used to output the answer to the user question based on the content of each target text block and the association between the contents of different target text blocks.

[0017] It can be understood that, by sorting and organizing the plurality of user questions and the plurality of target text blocks, the large language model can perform reasoning based on more complete information, and thus the reasoning effect of the large language model is significantly improved.

[0018] In a possible implementation, the target input sequence is obtained based on the user question and the plurality of target text blocks, including: for the plurality of target text blocks belonging to the same document, the plurality of target text blocks are sorted according to positions of the plurality of target text blocks in the same document indicated by respective source information of the plurality of target text blocks, to obtain sorted text blocks; and the target input sequence is obtained based on the user question and the sorted text blocks.

[0019] It can be understood that the plurality of target text blocks under the same document originally have a sequence, and the plurality of target text blocks that are out of order are restored to the original order according to the source information, thereby reducing the overhead of the large language model in understanding the text, avoiding misleading the large language model by the disordered sequence, and thereby improving the reasoning effect of the large language model.

[0020] In a possible implementation, the target condition includes: the evaluation result of the current retrieval result output by the large language model indicates that the next round of retrieval is not needed; or, the number of retrieval rounds reaches a set number of rounds; or, the time cost of answering the user question reaches a set range.

[0021] It can be understood that by setting the target condition, the retrieval process can be reasonably ended, and excessive retrieval or a too long time cost of the retrieval process can be avoided, thereby prolonging the overall time cost of reasoning.

[0022] In a possible implementation, the retrieval condition of the first round of retrieval further includes a first threshold range, and the retrieval result of the first round of retrieval includes a second text block, the second text block being a text block in the knowledge base that has a similarity to the user question meeting the first threshold range.

[0023] It can be understood that in the first round of retrieval, by setting the first threshold range, a text block highly similar to the user question can be screened from the knowledge base, thereby serving as one of the bases for subsequent rounds of retrieval, and helping to improve the accuracy of retrieval.

[0024] In a second aspect, an example of the present application provides a question and answer device, which is configured to execute any of the question and answer methods provided in the first aspect.

[0025] In a possible implementation, the example of the present application can divide the question and answer device into functional modules according to the method provided in the first aspect. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. For example, the example of the present application can divide the question and answer device into a retrieval module, an obtaining module, and the like according to functions. The possible technical solutions and beneficial effects of each functional module described above can be referred to the technical solutions provided in the first aspect or the corresponding possible implementation thereof, which will not be described here.

[0026] Thirdly, embodiments of this application provide a computing device that includes a management controller and a memory, with a processor coupled to the memory; the memory is used to store computer instructions that are loaded and executed by the processor to enable the computing device to implement the question-and-answer method as described above.

[0027] Fourthly, embodiments of this application provide a computer-readable storage medium storing at least one computer program instruction, which is loaded and executed by a management controller in a computing device to implement the question-and-answer method as described above.

[0028] Fifthly, embodiments of this application provide a computer program product including computer instructions stored in a computer-readable storage medium. A management controller of a computing device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computing device to perform the question-and-answer methods provided in the various optional implementations of the first aspect described above.

[0029] For a detailed description of the second to fifth aspects and their various implementations in the embodiments of this application, please refer to the detailed description in the first aspect and its various implementations; and for a detailed description of the beneficial effects of the second to fifth aspects and their various implementations, please refer to the beneficial effect analysis in the first aspect and its various implementations, which will not be repeated here.

[0030] These or other aspects of the embodiments of this application will become more apparent in the following description. Attached Figure Description

[0031] Figure 1 This application provides an architectural diagram of an intelligent question-answering system as an embodiment of the present application.

[0032] Figure 2 This is a schematic diagram of the hardware structure of a computing device provided in an embodiment of this application;

[0033] Figure 3 A schematic diagram illustrating the construction of a knowledge base, provided as an embodiment of this application;

[0034] Figure 4 for Figure 3 The illustrated embodiment is a schematic diagram of internal data of a knowledge base.

[0035] Figure 5 for Figure 3 The illustrated embodiment is a schematic diagram of a document splitting method.

[0036] Figure 6 A flowchart illustrating a question-and-answer method provided in an embodiment of this application;

[0037] Figure 7 For Figure 6 A schematic diagram of a multi-round search according to an embodiment of the present application is shown in FIG. 1.

[0038] Figure 8 A schematic diagram of a structure of a question-answering device according to an embodiment of the present application is shown in FIG. 2. DETAILED DESCRIPTION

[0039] To make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the embodiments of the present application will be further described in detail below with reference to the drawings.

[0040] In the present document, "a plurality of" means two or more. "And / or" describes the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can mean that there are three cases of A alone, A and B together, and B alone. The character " / " generally represents an "or" relationship between the associated objects before and after it.

[0041] In addition, in the description of the embodiments of the present application, "a plurality of" means two or more, unless otherwise specified. "At least one of" or the like means any combination of the items, including a single item or a plurality of items. For example, at least one of a, b, or c can mean a, b, c, a-b, a-c, b-c, or a-b-c, where a, b, and c can be a single item or a plurality of items.

[0042] In addition, in order to facilitate the clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, "first", "second", and the like are used to distinguish the same items or similar items with basically the same function and effect. Those skilled in the art can understand that "first", "second", and the like do not limit the quantity and execution order, and "first", "second", and the like do not necessarily mean different. At the same time, in the embodiments of the present application, "exemplary" or "for example" is used to represent an example, illustration or description. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as more preferred or more advantageous than other embodiments or design schemes. Rather, the use of "exemplary" or "for example" is intended to present the relevant concept in a specific manner, for understanding.

[0043] First, the application scenarios of the embodiments of the present application are exemplarily introduced.

[0044] The question and answer method provided by the embodiments of the present application can be applied to the intelligent question and answer scene. At present, the answer to the user question is not perfect and the quality is low. Therefore, the present application provides a question and answer method. On the basis of supporting multi-round retrieval, the source information can be used to directly locate the position of the target text block in the original document, and / or the graph relationship between different documents is combined to determine different documents with higher correlation degree. The correlation relationship between different text blocks in the same document and between different documents is fully utilized to improve the accuracy of retrieval and the answer quality of the large language model.

[0045] For example, the user opens an intelligent question and answer application program on a terminal device (for example, a mobile phone or a computer) and inputs a question. After receiving the user question, the remote server performs multi-round retrieval in the pre-constructed knowledge base, and then uses the retrieved information to obtain the answer to the user question and returns the answer to the terminal device.

[0046] In some possible embodiments, the method comprises: performing multi-round retrieval on the knowledge base based on the user question, the retrieval result of each round of retrieval comprising at least one target text block, until the plurality of target text blocks retrieved meet a target condition; and obtaining the answer to the user question based on the plurality of target text blocks; wherein the knowledge base comprises a plurality of text blocks, the text blocks being associated with respective source information, the source information being used to indicate the document to which the text block belongs or the position of the text block in the document; the retrieval condition of the first round of retrieval comprising the user question; and the retrieval condition of at least one round of retrieval in the remaining rounds of retrieval comprising the source information associated with at least one target text block that has been retrieved. Since the source information associated with the target text block that has been retrieved is used in the retrieval process, the context of the target text block originally in the same document or the text block in different documents but highly correlated can be accurately obtained, so that the perfection degree of the retrieval result can be accurately and effectively improved, and the reasoning effect of the model is improved.

[0047] Secondly, the system architecture of the embodiments of the present application is exemplarily introduced.

[0048] The question and answer method provided by the embodiments of the present application can be applied to the system architecture as shown in Figure 1 The system architecture as shown in Figure 1 is a schematic diagram of the architecture of an intelligent question and answer system provided by the embodiments of the present application. Figure 1 The intelligent question and answer system 1000 in the system architecture as shown in

[0049] Specifically, the retrieval module 1100 can be used to perform multi-round retrieval on the knowledge base based on the user question, and each round of retrieval result includes at least one target text block until the retrieved target text blocks meet the target condition. During the multi-round retrieval, the retrieval module 1100 can reasonably use the source information of the text blocks to provide a rich and complete retrieval result for the large language model.

[0050] The acquisition module 1200 can be used to acquire the answer to the user question based on the target text blocks retrieved by the retrieval module 1100, for example, combining the target text blocks with the user question to form a complete prompt word input into the large language model to obtain the answer output by the large language model.

[0051] The knowledge base 1300 stores at least a plurality of text blocks (which can specifically include text information of the text blocks and vector information corresponding to the text blocks). Optionally, the knowledge base 1300 also stores a graph relationship indicating the association relationship between different documents.

[0052] It is worth noting that each text block in the embodiments of the present application is also associated with respective source information (which can be stored in the knowledge base or other locations, which is not limited in the present application).

[0053] Figure 1 The intelligent question answering system 1000 shown can be deployed on one or more computing devices to implement the question answering method provided in the embodiments of the present application. When Figure 1 When the intelligent question answering system 1000 shown is deployed on multiple computing devices, the multiple computing devices can respectively perform different steps in the method to cooperatively implement the method.

[0054] The following will take Figure 1 The intelligent question answering system 1000 shown is deployed on one computing device as an example to describe the system architecture of the embodiments of the present application. Specifically, the computing device can be a server, and its specific hardware structure can refer to the computing device 3000 in Figure 2 Figure 2 The terminal device 4000 capable of communicating with the computing device 3000 is also shown in the figure, and the two devices are used together to provide the question answering method proposed in the embodiments of the present application.

[0055] Specifically, Figure 2 The computing device 3000 shown at least includes a memory 3010, a processor 3020, and a bus 3030. The terminal device 4000 can be a computing device including necessary peripheral devices (not shown in the figure), which include but are not limited to a touch screen, a keyboard, a mouse, a microphone, etc. Alternatively, the terminal device 4000 can be a peripheral device of the computing device 3000. Figure 2 The computing device 3000 shown at least includes a memory 3010, a processor 3020, and a bus 3030. The terminal device 4000 can be a computing device including necessary peripheral devices (not shown in the figure), which include but are not limited to a touch screen, a keyboard, a mouse, a microphone, etc. Alternatively, the terminal device 4000 can be a peripheral device of the computing device 3000.​

[0056] The user inputs a question to be answered on the terminal device 4000, and the terminal device 4000 forwards the question input by the user to the computing device 3000, so that the computing device 3000 performs the question-answering method provided in the embodiments of the present application based on the question of the user.

[0057] The processor 3020 in the computing device 3000 can be used at least to perform multi-round retrieval on the knowledge base based on the question of the user, the retrieval result of each round of retrieval includes at least one target text block, until the plurality of target text blocks retrieved satisfy a target condition, and obtain an answer to the question of the user based on the plurality of target text blocks, and the like. The storage 3010 can be used to store the logic code corresponding to the question-answering method provided in the embodiments of the present application.

[0058] Optionally, the computing device 3000 can be a rack server, an entire cabinet server, and the like, and the computing device 3000 can also be a computer, a mobile phone terminal, a tablet computer, a notebook computer, a desktop computer, an all-in-one machine, a personal digital assistant (PDA), an ultra-mobile personal computer (UMPC), and the like.

[0059] Optionally, the storage 3010 can include a random access memory (RAM), a read-only memory (ROM), and the like, wherein the storage 3010 can run a necessary operating system in the RAM thereof, and the retrieval module, the obtaining module, and the like, used to perform the question-answering method provided in the present application.

[0060] Optionally, the processor 3020 can be a central processing unit (CPU) or other general-purpose processor, an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processing (DSP), or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware unit, and the like. The general-purpose processor can be a microprocessor or any conventional processor, and the like.

[0061] Optionally, the bus 3030 can be a peripheral component interconnect (PCI) bus or other bus, and the application is not limited to the type of bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 2 only one line is used, but it does not mean that there is only one bus or only one type of bus. The bus 3030 can include a path for transmitting information between various components of the computing device 3000 (for example, the memory 3010 and the processor 3020).

[0062] It is worth noting that a server in the general sense does not have a display screen. For example, when the computing device 3000 described above is a server, the computing device 3000 needs to cooperate with the terminal device 4000 to implement the method of the application.

[0063] However, in some special scenarios, the computing device has a display screen and a user input interface, so that the computing device can directly respond to / receive the input of the user and display information, such as displaying various application pages. Such a computing device can be understood as an electronic device that integrates the computing device 3000 and the terminal device 4000 described above.

[0064] The system architecture and application scenarios described in the embodiments of the application are used to more clearly illustrate the technical solutions of the embodiments of the application, and do not constitute a limitation on the technical solutions provided by the embodiments of the application. Those skilled in the art can know that, as the system architecture evolves and new business scenarios appear, the technical solutions provided by the embodiments of the application are also applicable to similar technical problems.

[0065] In the embodiments of the application, the inference capability of a large language model (LLM) can be fully utilized to assist in searching and inferring answers to user questions.

[0066] The large language model is a model composed of artificial neural networks with a large number of parameters. Users can communicate with the large language model through a computing platform to obtain relevant information. For example, a user can input the question "What is the speed of light?" and the large language model outputs the answer "The speed of light is about 300,000 kilometers per second." Through similar question and answer dialogues, the large language model can help users learn more knowledge and improve work efficiency.

[0067] In addition, the embodiments of the application also involve using retrieval-augmented generation (RAG) technology to further improve the inference effect of the large language model and improve the answer quality.

[0068] RAG (Reference-Based Knowledge) is a technique that uses a pre-built external knowledge base to supplement the context of a large language model and generate answers. Specifically, RAG combines parametric and non-parametric external knowledge within the large language model, reducing the probability of incorrect answers. By customizing and updating the knowledge base used by RAG, the accuracy of large language models in specific domains can be enhanced. Furthermore, by showing users the sources of the answers, the transparency and user trust in the output of the large language model can be increased.

[0069] RAG technology can be used to improve the reasoning performance of large language models, but the application of RAG technology relies on a pre-built high-quality knowledge base. Therefore, in order to better understand the question-answering method proposed in the embodiments of this application, the knowledge base construction process involved in the embodiments of this application will be introduced below.

[0070] See Figure 3 , Figure 3 This is a schematic diagram illustrating the construction of a knowledge base according to an embodiment of this application, wherein, Figure 3 The knowledge base can be the aforementioned knowledge base 1300.

[0071] like Figure 3 As shown, in summary, the embodiments of this application first segment the documents to be entered into the database to obtain multiple text blocks (such as...). Figure 3 Only text blocks 1-3 are shown (in reality, more may be included), and then they are stored (at the same time, each text block can be vectorized, and the resulting vectors are also stored) to obtain a knowledge base, in which each text block is also associated with its own source information.

[0072] In one possible implementation, when segmenting documents to be imported into the database, they can be segmented according to a set length range. Furthermore, punctuation marks such as periods and question marks are used as segmentation points during segmentation. Thus, compared to traditional methods of segmenting the original document to a fixed length, the segmentation method used in this embodiment ensures that a complete sentence is not split in the middle, thereby guaranteeing semantic integrity. Moreover, before inference, the segmented text blocks of varying lengths can be padded (using specified characters) to ensure consistency in data volume across batches, which helps improve inference efficiency.

[0073] In one possible implementation, when splitting documents to be added to the database, directory nodes can also be used as splitting points (usually, the content under different directory nodes has certain differences), thereby ensuring that the text content under different directory nodes will not be split into the same text block.

[0074] The source information of the text block is described below. In a document, there is usually a directory composed of multiple directory nodes (for example, common first-level headings, second-level headings, and the like). Through each directory node, the theme of the corresponding text content can be understood. In other words, the text content under the same directory node (which can be referred to as the minimum text in the embodiment of the application) can be considered as highly relevant.

[0075] In the embodiment of the application, the source information associated with each text block can be divided into three different levels, specifically including: a document level, a minimum text level, and a text block level.

[0076] Among them, the source information of the document level can uniquely specify the document to which the text block belongs; the source information of the minimum text level can uniquely specify the directory node to which the text block belongs; and the source information of the text block level can uniquely specify which segment of the multiple text blocks under the same directory node the text block is.

[0077] For example, the source information of the document level can be composed of a document number or a document name (which needs to ensure that there is no duplicate document name in the knowledge base). The source information of the minimum text level can be composed of at least a directory node number (which can also include the source information of the document level). The source information of the text block level can be composed of at least a serial number determined based on the order of the multiple text blocks in the original document (which can also include the source information of the document level and the source information of the minimum text level).

[0078] In this case, as Figure 4 indicated, Figure 4 the embodiment shown in Figure 3 is a schematic diagram of the internal data of the knowledge base, which shows text block 1-text block 3 and their associated source information 1-source information 3. The source information can specifically include a document number, source information of the minimum text level, and source information of the text block level. In this way, during retrieval, the computing device can directly locate to a specific position (for example, which document it belongs to) according to the source information of the text block, and then input the directory corresponding to the document into the large language model to guide the large language model to decide the next round of retrieval conditions. Alternatively, the context in the minimum text where the text block is located can be obtained directly according to the source information of the text block, and a more perfect retrieval result can be provided to the large language model to improve the reasoning effect.

[0079] As shown on the right side of Figure 4 , the knowledge base in the embodiment of the application can also store specific directory information and be associated with each document (document number), as shown in Figure 4 , where directory 1 corresponds to document number 1, and directory 2 corresponds to document number 2.

[0080] It is worth noting that,Figure 4 This is just one possible example; text blocks 1 through 3 could all be segments from document number 1 (or segments from different documents), while the knowledge base contains many more text blocks and document numbers. Figure 4 Not shown.

[0081] In addition, the knowledge base in this application embodiment can also store graph relationships indicating the association between different documents. This application does not limit how to generate graph relationships.

[0082] In other words, the knowledge base in this embodiment of the application can store various types of information such as text blocks, directories, vectors corresponding to text blocks, and graph information between different documents.

[0083] See details Figure 5 , Figure 5 for Figure 3 The illustrated embodiment is a schematic diagram of a document splitting process. Figure 5 This shows how to obtain the directory nodes in document 1 from document 1, and the multiple text blocks obtained by segmenting document 1. Figure 5 Only text blocks 1-3, the vectors corresponding to each text block (vector 1-vector 3), and the graph relationships obtained based on document 1 and document 2 are shown. Figure 5 This is just one possible example, and only a portion of the data in the knowledge base is shown.

[0084] In some feasible embodiments, each text block, the vector corresponding to each text block, and the source information associated with each text block can also be stored in different databases.

[0085] For example, the knowledge base stores only the text and the vector corresponding to each text, while the source information is stored in other databases (specifically, tables). Then, when a query is needed, the source information corresponding to the text block can be determined by looking up the table, so as to realize the further retrieval process.

[0086] Based on the knowledge base constructed above and the source information associated with each text box, the embodiments of this application can achieve an efficient and accurate multi-round retrieval process.

[0087] The question-and-answer method provided in the embodiments of this application will be explained and described below with reference to the accompanying drawings. This method is applicable to the above-mentioned... Figure 2 The computing device 3000 shown is described in detail below. Figure 6 As shown, Figure 6 A flowchart of a question-and-answer method provided in this application embodiment specifically includes:

[0088] S110, the computing device performs multiple rounds of retrieval of the knowledge base based on the user's question.

[0089] wherein the search results of each round of searching include at least one target text block until the plurality of target text blocks retrieved satisfy a target condition.

[0090] In the embodiments of the present application, a plurality of different target conditions can be set, including: the evaluation result of the large language model output for the current search result indicates that the next round of searching is not needed; or, the number of search rounds reaches a set number of rounds; or, the time consumption of answering the user question reaches a set range.

[0091] It is worth noting that in the process of multi-round searching, in the case of reaching any one of the above target conditions, the computing device can immediately end the searching process, and then execute step S120, thereby avoiding excessive searching and keeping the overall time consumption of the searching process within an acceptable range.

[0092] The multi-round searching process is explained and described below, and can be specifically referred to Figure 7 , Figure 7 for a schematic diagram of the multi-round searching involved in the embodiments shown in Figure 6 .

[0093] Specifically, as shown in Figure 7 , the computing device first retrieves, based on the obtained user question, text blocks (i.e., target text blocks) from the knowledge base that have a similarity to the question within a first threshold range, wherein the first threshold range can be customized by the user.

[0094] Since there is no available source information for the user question in the first round of searching, methods such as keyword matching, vector similarity searching, or hybrid searching can be selected to retrieve text blocks highly similar to the user question (within the first threshold range) from the knowledge base.

[0095] And in the case where the first round of searching obtains a plurality of text blocks (or first round candidate text blocks, second text blocks) within the first threshold range, the plurality of first round candidate text blocks can be further scored, for example using a reranker model, and according to the ranking result of the model, one or more most relevant text blocks are selected for subsequent searching process.

[0096] In some feasible embodiments, the computing device can also directly input the text blocks obtained in the first round of searching to the large language model, and if the large language model infers that the current search result is perfect enough to answer the user question (satisfies the target condition), the computing device can also immediately execute step S120, thereby improving the inference efficiency.

[0097] To improve the accuracy and completeness of search results, after the initial search, the computing device can continue to perform one or more rounds of searches in the knowledge base based on the results obtained in the initial search (i.e., corresponding searches). Figure 7 The process of retrieval in subsequent rounds is similar, with each round based on the source information associated with the search results from the previous round. The specific retrieval method used in each round can be based on semantic vector similarity, keyword matching, or a combination of both, or it can be a targeted retrieval method based on specified directory nodes.

[0098] Therefore, the following explanation and illustration will take any subsequent round of retrieval after the first round as an example. In the following embodiments, the first text block refers to any target text block that has already been retrieved.

[0099] Based on source information at different levels, this application provides at least the following possible implementations:

[0100] In the first implementation, the source information of the first text block is used to indicate that the position of the first text block in the first document is under the first directory node. At least one round of retrieval process in each subsequent round of retrieval includes: using the source information of the first text block as the retrieval condition, obtaining the third text block in the knowledge base that is associated with the source information of the first text block. The third text block is a text block other than the first text block under the first directory node of the first document.

[0101] Specifically, as mentioned above, the content of text under the same directory node is highly related. Therefore, the computing device can use the source information of the first text block as the retrieval condition to retrieve other text blocks (the third text block) that are located under the same directory node (highly related) as the first text block in a targeted and accurate manner.

[0102] Considering that the text content under the same directory node may be very large, causing an excessive burden on the reasoning task, it is also possible to further combine the source information of the text block level for retrieval to obtain an acceptable range of context length, thereby achieving a balance between the completeness of the retrieval results and the burden of the reasoning task.

[0103] For example, the source information of the first text block indicates that the first text block is located under the 5th directory node in "Document 0001". Therefore, based on this source information, the computing device can directly obtain other text blocks that are also located under the 5th directory node in "Document 0001".

[0104] In some possible embodiments, due to the existence of a large number of text blocks under the 5th directory node, the computing device further identifies the 6th text block under the 5th directory node in combination with the position of the first text block in the "document 0001". In turn, the computing device acquires the 3rd text block-5th text block and the 7th text block-9th text block under the 5th directory node in the "document 0001".

[0105] In the second implementation, the source information of the first text block is used to indicate that the first text block belongs to the first document, and each round of retrieval process of the at least one round of retrieval further includes: taking the source information of the first text block as a retrieval condition, and acquiring, based on a pre-generated graph relationship, a fourth text block in the knowledge base that is associated with the source information of the first text block, where the graph relationship is used to indicate an association relationship between different documents in the knowledge base, and the fourth text block is a text block in a second document that is associated with the first document in the knowledge base.

[0106] Specifically, in combination with the source information of the document level, the computing device can fully utilize the graph relationship indicating the association relationship between different documents in the knowledge base to acquire text blocks in different documents that are highly relevant to the text block in the first text block.

[0107] For example, the source information of the first text block indicates that the first text block is in the "document 0005". In turn, the computing device determines, according to the source information and the pre-generated graph relationship, that the "document 0003", the "document 0007" and the "document 0005" in the knowledge base are highly relevant, and then the computing device can retrieve text blocks highly relevant to the first text block from the text content corresponding to the "document 0003" and the "document 0007" to obtain more perfect retrieval results.

[0108] In the third implementation, the source information of the first text block is used to indicate that the first text block belongs to the first document, and each round of retrieval process of the at least one round of retrieval further includes: inputting the user question and the directory information of the first document into a large language model to obtain at least one second directory node output by the large language model; and taking the second directory node as a retrieval condition to acquire a third text block in the knowledge base that is associated with the second directory node of the first document.

[0109] The third implementation manner fully utilizes the reasoning capability of the large language model, and inputs the user question and the directory information of the first document into the large language model, so that the large language model decides how to retrieve in the next round. In this way, compared with the traditional multi-round retrieval manner (without inputting the directory information into the large language model), in the question and answer method provided in the present application, the large language model obtains the directory information that is clear in direction when making a decision. Further, the new retrieval condition given by the large language model in the embodiment of the present application is more specific and clear in direction, rather than searching for simple keywords globally in the knowledge base, so that the retrieval efficiency and accuracy can be significantly improved.

[0110] In addition to utilizing the large language model to reason the second directory node, in the embodiment of the present application, at least the retrieval result obtained in the last time can be input into the large language model after each round of retrieval ends, to obtain the retrieval condition of the next round of retrieval process output by the large language model, and the retrieval condition of the next round of retrieval process includes at least one source information associated with the retrieved target text block.

[0111] Further, the graph relationship of the first document in which the first text block is located can also be input into the large language model, so that the large language model not only masters the internal structure of the first document in which the first text block is located, but also knows other document information in the knowledge base that is highly associated with the first document, so that the large language model forms a clear cognition of the internal and external association of the first document, and further can output the new retrieval condition that is clear in direction, thereby improving the accuracy and efficiency of retrieval.

[0112] For example, the computing device inputs the source information of the first text block into the large language model, that is, the position of the first text block in the "document 0010" is under the third directory node, which is the second text block, the user question 1: what is the speed of light, the 10 directory information of the "document 0010", and the graph relationship with the "document 0010". Further, the computing device obtains the output of the large language model: retrieving the fourth directory node in the "document 0010" and the text content associated with the "document 0011". Further, the computing device retrieves the associated third text block according to the foregoing retrieval condition.

[0113] Through the above various possible implementation manners, the computing device can efficiently perform multi-round retrieval on the knowledge base until a perfect retrieval result is obtained (or any one of the foregoing target conditions is met, corresponding Figure 7 The last part), and then performs the subsequent steps. Compared with the traditional single retrieval and the method of multiple retrievals through simple keywords (each time searching for different keywords globally in the knowledge base), the multi-round retrieval process in the embodiment of the present application is more directional, and more perfect and accurate retrieval results can be obtained with higher efficiency.

[0114] S120, the computing device obtains an answer to the user question based on the plurality of target text blocks.

[0115] In this step, the computing device can obtain a plurality of high-quality target text blocks through step S110, and then needs to input these target text blocks into the large language model to obtain the answer to the user question inferred by the large language model.

[0116] In a possible implementation, a target input sequence is obtained based on the user question and the plurality of target text blocks; the target input sequence is input into the large language model to obtain an answer output by the large language model; the large language model is used to output the answer to the user question based on the content of each target text block and the connection between the contents of different target text blocks.

[0117] Specifically, the computing device can appropriately adjust and organize the user question and the plurality of target text blocks, such as applying a preset template: "the user question is:...; the reference material is:...", and inputting the user question and the plurality of target text blocks into the corresponding positions to obtain a target input sequence, and then inputting the target input sequence into the large language model to obtain an answer output by the large language model.

[0118] In a possible implementation, for the plurality of target text blocks belonging to the same document, the computing device can sort the plurality of target text blocks according to the positions of the plurality of target text blocks in the same document indicated by the source information of the plurality of target text blocks to obtain sorted text blocks; and obtain a target input sequence based on the user question and the sorted text blocks.

[0119] Specifically, the plurality of target text blocks under the same document originally have a sequence, and since each target text block in the embodiment of the present application can be associated with source information, the computing device can restore the out-of-order plurality of target text blocks to the original order according to the source information, so that the semantics of the text content is smooth, the large language model is easier to understand, and the large language model is prevented from being misled by the chaotic order, thereby improving the inference effect of the large language model.

[0120] For example, the computing device obtains four target text blocks through multiple rounds of retrieval, which are: the second text block under the second directory node in "document 0100", the third text block under the second directory node in "document 0100", the first text block under the second directory node in "document 0100", and the third text block under the seventh directory node in "document 0200". Then, the computing device restores the three text blocks under "document 0100" to the original order, and inputs them together with the text block under "document 0200" and the user question into the large language model to obtain an answer to the user question output by the large language model.

[0121] By performing steps S110-S120, the computing device can accurately retrieve the knowledge base based on the user question and the retrieval results of each round, gradually improving the completeness and accuracy of the retrieval results, and further improving the reasoning effect of the large language model to obtain more accurate answers.

[0122] The above mainly introduces the scheme of the embodiments of the present application from the method aspect. It can be understood that the computing device comprises at least one of the corresponding hardware structure and software module for implementing each function. Those skilled in the art should easily realize that the units and algorithm steps of each example described in combination with the embodiments disclosed in the present application can be realized in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed in the form of hardware or computer software driven hardware depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of the present application.

[0123] The embodiments of the present application can divide the functional units of the question and answer device according to the above method examples. For example, each functional unit can be divided according to each function, or two or more functions can be integrated in one processing unit. The integrated unit can be realized in the form of hardware or software functional unit. It should be noted that the division of units in the embodiments of the present application is illustrative, and is only a logical functional division. Actual implementation can have another division method.

[0124] An exemplary question and answer device provided by the embodiments of the present application is shown in the following table. Figure 8 An exemplary question and answer device provided by the embodiments of the present application is shown in the following table. Figure 8 The question and answer device 800 shown can be applied to a computing device, or the question and answer device 800 can be a computing device. The question and answer device 800 comprises:

[0125] The retrieval module 810 is configured to perform multi-round retrieval on the knowledge base based on the user question, and the retrieval result of each round of retrieval comprises at least one target text block, until the plurality of target text blocks retrieved satisfy the target condition.

[0126] The acquisition module 820 is configured to acquire the answer to the user question based on the plurality of target text blocks; wherein the knowledge base comprises a plurality of text blocks, and each text block is associated with respective source information, and the source information is used to indicate the document to which the text block belongs or the position of the text block in the document; the retrieval condition of the first round of retrieval comprises the user question; and the retrieval condition of at least one of the remaining each round of retrieval process comprises the source information associated with at least one target text block that has been retrieved.

[0127] For example, in combination with Figure 6 The retrieval module 810 can be configured to perform S110 as shown in Figure 6 The acquisition module 820 can be configured to perform S120 as shown in Figure 6

[0128] In a possible implementation, the source information of the first text block is used to indicate that the first text block is under the first directory node in the first document, and the first text block is any target text block that has been retrieved; the retrieval module 810 is further configured to acquire, as a retrieval condition, a third text block associated with the source information of the first text block in the knowledge base, the third text block being a text block under the first directory node of the first document and other than the first text block.

[0129] In a possible implementation, the source information of the first text block is used to indicate that the first text block belongs to the first document; the first text block is any target text block that has been retrieved; the retrieval module 810 is further configured to input the user question and the directory information of the first document into the large language model to obtain at least one second directory node output by the large language model; and acquire, as a retrieval condition, a third text block associated with the second directory node of the first document in the knowledge base.

[0130] In a possible implementation, the source information of the first text block is used to indicate that the first text block belongs to the first document; the first text block is any target text block that has been retrieved; the retrieval module 810 is further configured to acquire, as a retrieval condition, a fourth text block associated with the source information of the first text block in the knowledge base based on a pre-generated graph relationship, wherein the graph relationship is used to indicate an association relationship between different documents in the knowledge base, and the fourth text block is a text block in a second document associated with the first document in the knowledge base.

[0131] In a possible implementation, after each round of retrieval ends, at least the latest obtained retrieval result is input into the large language model to obtain a retrieval condition for a next round of retrieval process output by the large language model, and the retrieval condition for the next round of retrieval process includes source information associated with at least one target text block that has been retrieved.

[0132] In a possible implementation, the acquisition module 820 is further configured to obtain a target input sequence based on the user question and the plurality of target text blocks; input the target input sequence into the large language model to obtain an answer output by the large language model; and the large language model is used to output an answer to the user question based on the content of each target text block and the association between the contents of different target text blocks.

[0133] ​In a possible implementation, the acquisition module 820 is further configured to, for the plurality of target text blocks belonging to the same document, sort the plurality of target text blocks according to positions of the plurality of target text blocks in the same document indicated by respective source information of the plurality of target text blocks, to obtain sorted text blocks; and obtain the target input sequence based on the user question and the sorted text blocks.

[0134] In a possible implementation, the target condition includes: an evaluation result of the large language model output for the current retrieval result indicating that the next round of retrieval is not needed; or, the number of retrieval rounds reaching a set number of rounds; or, a time cost of answering the user question reaching a set range.

[0135] In a possible implementation, the retrieval condition of the first round of retrieval further includes a first threshold range, and the retrieval result of the first round of retrieval includes a second text block, the second text block being a text block in the knowledge base that has a similarity to the user question conforming to the first threshold range.

[0136] As a feasible example, the question and answer device 800 provided in the present application is implemented through a software module. For example, the software module can be provided to users in a cloud service subscription mode, and users can select different subscription levels according to needs. For another example, the software module can also provide enterprise-level customized services with professional domain customization, interface personalization, and expansion functions according to the needs of users or enterprises.

[0137] In addition, the question and answer device 800 provided in the present application can also be made into a value-added service to provide users, which is not limited in the present application. When the question and answer device 800 is implemented through a software module, the question and answer device 800 can also be embedded into a retrieval tool or other intelligent question and answer systems.

[0138] In the above embodiments, all or part can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, the computer instructions generate all or part of the processes or functions in the embodiments of the present application. The computer can be a general purpose computer, a special purpose computer, a computer network or other programmable apparatus. The computer instructions can be stored in a computer readable storage medium or transferred from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions can be transferred from one website, computer, server or data center to another website, computer, server or data center through wired (for example, coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (for example, infrared, wireless, microwave, etc.) mode. The computer readable storage medium can be any available medium accessible by a computer or a data storage device such as a server, data center, etc. integrated with one or more available media. The available media can be magnetic media (for example, floppy disk, magnetic disk, magnetic tape), optical media (for example, digital video disc (DVD)) or semiconductor media (for example, solid state drive (SSD)) and the like.

[0139] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above division of functional modules is taken as an example, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device and unit described above can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

[0140] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the modules or units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the shown or discussed each other can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0141] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0142] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present alone, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0143] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: a flash memory, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk or an optical disk, and various media that can store program codes.

[0144] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or replacements within the technical scope disclosed in the present application should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A question and answer method, characterized by, The method comprises: based on the user question, a plurality of rounds of retrieval are performed on the knowledge base, and each round of retrieval comprises at least one target text block until the plurality of target text blocks retrieved meet a target condition; an answer to the user question is obtained based on the plurality of target text blocks; wherein the knowledge base comprises a plurality of text blocks, and each text block is associated with respective source information, the source information being used to indicate a document to which the text block belongs or a position of the text block in the document; a retrieval condition of a first round of retrieval comprises the user question; and a retrieval condition of at least one of the remaining rounds of retrieval comprises source information associated with at least one target text block that has been retrieved.

2. The method of claim 1, wherein, The source information of the first text block is used to indicate that the position of the first text block in the first document is under a first directory node, and the first text block is any target text block that has been retrieved; the at least one of the remaining rounds of retrieval comprises: using the source information of the first text block as a retrieval condition, a third text block associated with the source information of the first text block in the knowledge base is obtained, and the third text block is a text block under the first directory node of the first document other than the first text block.

3. The method of claim 1, wherein, The source information of the first text block is used to indicate that the first text block belongs to the first document; and the first text block is any target text block that has been retrieved; the at least one of the remaining rounds of retrieval comprises: inputting the user question and directory information of the first document into a large language model to obtain at least one second directory node output by the large language model; using the second directory node as a retrieval condition, a third text block associated with the second directory node of the first document in the knowledge base is obtained.

4. The method of claim 1, wherein, The source information of the first text block is used to indicate that the first text block belongs to the first document; and the first text block is any target text block that has been retrieved; the at least one of the remaining rounds of retrieval comprises: using the source information of the first text block as a retrieval condition, a fourth text block associated with the source information of the first text block in the knowledge base is obtained based on a pre-generated graph relationship, wherein the graph relationship is used to indicate an association relationship between different documents in the knowledge base, and the fourth text block is a text block in a second document associated with the first document in the knowledge base.

5. The method according to any one of claims 1-4, characterized in that, After each round of retrieval ends, at least the retrieval result obtained in the last time is input into a large language model to obtain a retrieval condition of a next round of retrieval process output by the large language model, and the retrieval condition of the next round of retrieval process comprises source information associated with the at least one target text block that has been retrieved.

6. The method according to any one of claims 1-5, characterized in that, The answer to the user question is obtained based on the plurality of target text blocks, comprising: a target input sequence is obtained based on the user question and the plurality of target text blocks; the target input sequence is input into a large language model to obtain an answer output by the large language model; and the large language model is used to output an answer to the user question based on the content of each target text block and the relationship between the contents of different target text blocks.

7. The method of claim 6, wherein, The target input sequence is obtained based on the user question and the plurality of target text blocks, and the obtaining comprises: For the plurality of target text blocks belonging to the same document, the plurality of target text blocks are sorted according to positions of the plurality of target text blocks in the same document indicated by respective source information of the plurality of target text blocks, to obtain sorted text blocks; The target input sequence is obtained based on the user question and the sorted text blocks.

8. The method according to any one of claims 1 to 7, characterized in that, The target condition comprises: The evaluation result of the large language model on the current search result indicates that the next round of search is not needed; or, The number of search rounds reaches a set number of rounds; or The time cost for answering the user question reaches a set range.

9. The method according to any one of claims 1-8, characterized in that, The search condition of the first round of search further comprises a first threshold range, and the search result of the first round of search comprises a second text block, the second text block being a text block in the knowledge base that has a similarity with the user question conforming to the first threshold range.

10. A computing device, comprising: The computing device comprises a processor and a memory; the processor is coupled with the memory; the memory is used to store computer instructions, the computer instructions are loaded and executed by the processor to enable the computing device to implement the question and answer method according to any one of claims 1 to 9. The target input sequence is obtained based on the user question and the plurality of target text blocks, and the obtaining comprises: For the plurality of target text blocks belonging to the same document, the plurality of target text blocks are sorted according to positions of the plurality of target text blocks in the same document indicated by respective source information of the plurality of target text blocks, to obtain sorted text blocks; The target input sequence is obtained based on the user question and the sorted text blocks. The target condition comprises: The evaluation result of the large language model on the current search result indicates that the next round of search is not needed; or, The number of search rounds reaches a set number of rounds; or The time cost for answering the user question reaches a set range. The search condition of the first round of search further comprises a first threshold range, and the search result of the first round of search comprises a second text block, the second text block being a text block in the knowledge base that has a similarity with the user question conforming to the first threshold range. The computing device comprises a processor and a memory; the processor is coupled with the memory; the memory is used to store computer instructions, the computer instructions are loaded and executed by the processor to enable the computing device to implement the question and answer method according to any one of claims 1 to 9.

Images