Method and device for multi-document multi-paragraph search query enhancement generation

Abstract

The invention relates to the technical field of question retrieval, and discloses a method for enhancing generation of multi-document and multi-paragraph search query. Comprises: receiving user question information; disassembling the user question information into a plurality of sub-questions through a question disassembling model; calling a text search module or a network search module for each sub-question in sequence to respectively generate answer fragment information; summarizing all the answer fragment information through a summarizing module to obtain context information of the question information of the user, and generating summarized answer information according to the context information and all the answer fragment information; storing the summarized answer information and the corresponding user question information; the method has the advantage of being capable of processing multiple documents, multiple paragraphs and complex queries.

Description

Technical Field

[0001] The present invention relates to the field of question retrieval technology, and in particular to a method and device for enhanced generation of multi-document and multi-paragraph search queries. Background Art

[0002] In traditional information systems, retrieval enhancement generation technology is usually used for information retrieval and question answering. Retrieval enhancement generation technology is a hybrid technology architecture that combines information retrieval and natural language generation. Its purpose is to use the retrieved relevant information to enhance the generated text content, making it information-rich and capable of interacting with users. Traditional RAG technology performs well in processing simple, question-answering, single-document scenarios that do not require multiple paragraphs, multiple documents, and require summary and refinement, and can better find the corresponding answer from the document.

[0003] The components of traditional retrieval-enhanced generation technology include: information retrieval module, generation module and integrated architecture; traditional RAG technology is usually used in question-answering systems; limitations include dependence on retrieval results, such as if the retrieval module fails to find relevant information, the generation module will also be restricted; in addition, the ability to process multiple documents, multiple paragraphs or complex queries is also limited.

[0004] Therefore, a method and device for enhanced generation of multi-document and multi-paragraph search queries are provided to solve the above problems. Summary of the invention

[0005] The main purpose of the present invention is to solve the problems in the prior art, that is, traditional RAG technology is usually used in question-answering systems; there is a dependence on retrieval results, if the retrieval module fails to find relevant information, the generation module will also be restricted; the ability to query multiple documents, multiple paragraphs or complex queries is also limited, and so on.

[0006] A first aspect of the present invention provides a method for enhanced generation of multi-document multi-paragraph search queries, the method for enhanced generation of multi-document multi-paragraph search queries comprising:

[0007] Receive user problem information;

[0008] Decompose user question information into several sub-questions through the question decomposition model;

[0009] Calling the text search module or the network search module for each sub-question in turn to generate answer fragment information respectively;

[0010] All answer fragment information is summarized through the summary module to obtain the context information of the user's question information, and the summary answer information is generated based on the context information and all answer fragment information;

[0011] The summary answer information and the corresponding user question information are stored.

[0012] Optionally, it also includes: acquiring data information, establishing a database according to the field, the database including several levels of indexes, and storing the data information in the corresponding indexes.

[0013] Optionally, the decomposition process of the problem decomposition model includes:

[0014] Reorganize user questions and break them down into several sub-questions based on technical fields and answer generation methods;

[0015] The answer generation method includes: database text search generation and network search generation.

[0016] Optionally, calling a text search module or a network search module for each sub-question in sequence to generate answer fragment information includes:

[0017] For each sub-problem, a judgment is made to determine whether the sub-problem can be decomposed through the problem decomposition model. If so, the sub-problem is decomposed into several subordinate questions, and answer fragment information is generated for each subordinate question. If not, the answer fragment information is directly generated.

[0018] Optionally, generating answer fragment information for each subordinate question includes:

[0019] Determine whether the answer generation method for the subordinate question is applicable to database text search generation or network search generation; if applicable to database text search generation, call the text search module to generate answer fragment information; if applicable to network search generation, call the network search module to generate answer fragment information.

[0020] Optionally, the directly generated answer fragment information includes:

[0021] Determine whether the answer generation method of the sub-question is applicable to database text search generation or network search generation; if applicable to database text search generation, call the text search module to generate answer fragment information; if applicable to network search generation, call the network search module to generate answer fragment information.

[0022] A second aspect of the present invention provides a device for enhanced generation of multi-document multi-paragraph search queries, comprising:

[0023] A question receiving module is used to receive user question information;

[0024] The sub-question decomposition module is used to decompose the user question information into several sub-questions through the question decomposition model;

[0025] An answer fragment generation module is used to call the text search module or the network search module to generate answer fragment information for each sub-question in turn;

[0026] The answer information aggregation module is used to aggregate all the answer fragment information through the summary aggregation module to generate aggregated answer information;

[0027] The answer storage module is used to store the summary answer information and the corresponding user question information.

[0028] A third aspect of the present invention provides an electronic device, the electronic device comprising a memory and at least one processor, the memory storing instructions;

[0029] The at least one processor calls the instructions in the memory to enable the electronic device to execute the various steps of the method for enhanced generation of multi-document and multi-paragraph search queries as described above.

[0030] A fourth aspect of the present invention provides a computer-readable storage medium having instructions stored thereon, which, when executed by a processor, implement the various steps of the method for enhanced generation of multi-document and multi-paragraph search queries as described above.

[0031] In the technical solution of the present invention, user questions are combined with the context to achieve a final result that takes into account the user conversation context at the same time, and generates content that is closer to semantics and smoother; the present application can also classify document knowledge and divide it into multi-level indexes according to specific needs / scenarios, with one type of knowledge model responsible for one type of index, so that cross-document and cross-paragraph retrieval can be achieved to solve complex problems; the present application can achieve the ability for the entire device to use tools such as network search / calling internal system interfaces, as well as the ability to search for answers from the document knowledge base. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 A flow chart of a method for enhanced generation of multi-document multi-paragraph search queries provided by an embodiment of the present invention;

[0033] Figure 2 A schematic diagram of the structure of an apparatus for enhanced generation of multi-document and multi-paragraph search queries provided by an embodiment of the present invention;

[0034] Figure 3 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0035] An embodiment of the present invention provides a method for enhanced generation of multi-document and multi-paragraph search queries, including receiving user question information; decomposing the user question information into a number of sub-questions through a question decomposition model; calling a text search module or a network search module for each sub-question in turn to generate answer fragment information respectively; summarizing all the answer fragment information through a summary module to generate summary answer information; storing the summary answer information and the corresponding user question information. The present invention solves the problems in the prior art that traditional RAG technology is usually used in question-answering systems; there is a dependence on retrieval results, such as when the retrieval module fails to find relevant information, the generation module will also be restricted; the ability of multi-document, multi-paragraph or complex queries is also limited, and the like.

[0036] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0037] For ease of understanding, the specific process of the embodiment of the present invention is described below. Figure 1 The first embodiment of the method for enhanced generation of multi-document multi-paragraph search queries in the embodiment of the present invention includes:

[0038] Acquire data information, establish a database according to the field, the database includes several levels of indexes, and store the data information in the corresponding indexes;

[0039] Receive user problem information;

[0040] Decomposing the user question information into several sub-questions through the question decomposition model; the specific decomposition process includes: reorganizing the user question, and decomposing the reorganized user question into several sub-questions according to the technical field and the answer generation method; the answer generation method includes: database text search generation and network search generation;

[0041] Call the text search module or the network search module for each sub-question in turn to generate answer fragment information respectively; judge for each sub-question whether the sub-question can be decomposed by the question decomposition model, if so, decompose the sub-question into several subordinate questions, and generate answer fragment information for each subordinate question, if not, directly generate answer fragment information;

[0042] The generating of answer fragment information for each subordinate question includes:

[0043] Determine whether the answer generation method for the subordinate question is applicable to database text search generation or network search generation; if applicable to database text search generation, call the text search module to generate answer fragment information; if applicable to network search generation, call the network search module to generate answer fragment information.

[0044] The directly generated answer fragment information includes:

[0045] Determine whether the answer generation method of the sub-question is applicable to database text search generation or network search generation; if applicable to database text search generation, call the text search module to generate answer fragment information; if applicable to network search generation, call the network search module to generate answer fragment information.

[0046] All answer fragment information is summarized through the summary module to obtain the context information of the user's question information, and the summary answer information is generated based on the context information and all answer fragment information;

[0047] The summary answer information and the corresponding user question information are stored.

[0048] See also Figure 1 A second embodiment of the method for enhanced generation of multi-document multi-paragraph search queries in an embodiment of the present invention includes:

[0049] Acquire data information, establish a database according to the field, the database includes several levels of indexes, and store the data information in the corresponding indexes;

[0050] Receive user problem information;

[0051] Decomposing the user question information into several sub-questions through the question decomposition model; the specific decomposition process includes: reorganizing the user question, and decomposing the reorganized user question into several sub-questions according to the technical field and the answer generation method; the answer generation method includes: database text search generation and network search generation;

[0052] Call the text search module or the network search module for each sub-question in turn to generate answer fragment information respectively; judge for each sub-question whether the sub-question can be decomposed by the question decomposition model, if so, decompose the sub-question into several subordinate questions, and generate answer fragment information for each subordinate question, if not, directly generate answer fragment information;

[0053] The generating of answer fragment information for each subordinate question includes:

[0054] Determine whether the answer generation method of the subordinate question is applicable to database text search generation or network search generation; if applicable to database text search generation, call the text search module to generate answer fragment information; if applicable to network search generation, call the network search module to generate answer fragment information, summarize the answer fragment information of all subordinate questions belonging to the same sub-question, and generate sub-question answer fragment information;

[0055] The directly generated answer fragment information includes:

[0056] Determine whether the answer generation method of the sub-question is applicable to database text search generation or network search generation; if applicable to database text search generation, call the text search module to generate answer fragment information; if applicable to network search generation, call the network search module to generate answer fragment information.

[0057] The sub-question answer fragment information and other answer fragment information are summarized through the summary module to obtain the context information of the user's question information, and the summary answer information is generated according to the context information and all the answer fragment information;

[0058] The summary answer information and the corresponding user question information are stored.

[0059] The following is a specific example of this embodiment:

[0060] A user asked, "Please compare Real Madrid and Manchester United based on the team's recent results, player salaries, and team history";

[0061] First, user questions are broken down according to tools and knowledge bases, including sub-question A "Real Madrid's recent results", sub-question B "Manchester United's recent results", sub-question C "Real Madrid player salaries, team history", and sub-question D "Manchester United player salaries, team history".

[0062] Determine whether sub-question A, sub-question B, sub-question C, and sub-question D can be further split. If sub-question A and sub-question B cannot be further split, then determine sub-question A. Sub-question A needs to be answered through retrieval. Use a network search tool to search for sub-question A and generate an answer fragment. Determine sub-question B. Sub-question B needs to be answered through retrieval. Use a network search tool to search for sub-question B and generate an answer fragment.

[0063] Sub-problems C and D can be further split. According to the primary index and secondary index in the knowledge base, sub-problem C can be split into "Real Madrid player salary" and "Real Madrid team history". Sub-problem D can be split into "Manchester United player salary" and "Manchester United team history" in the same way.

[0064] Find the corresponding answer fragment from the knowledge base according to the decomposed sub-questions;

[0065] Aggregate the answer fragments in the knowledge base into the summary model to summarize the summary answer of "Real Madrid and Manchester United player salaries and history",

[0066] The answers obtained from the web search tool and the text search generated by the central control are summarized and pieced together to form a complete answer to the user's question "Please compare Real Madrid and Manchester United based on the team's recent results, player salaries, and team history";

[0067] Show the complete answer to the user and complete a full round of interaction with the user.

[0068] The above describes the method for enhancing the generation of multi-document and multi-paragraph search queries in the embodiment of the present invention. The following describes the device for enhancing the generation of multi-document and multi-paragraph search queries in the embodiment of the present invention. Figure 2 In the embodiment of the present invention, the device for enhancing the generation of multi-document and multi-paragraph search queries includes:

[0069] A question receiving module is used to receive user question information;

[0070] The sub-question decomposition module is used to decompose the user question information into several sub-questions through the question decomposition model;

[0071] An answer fragment generation module is used to call the text search module or the network search module to generate answer fragment information for each sub-question in turn;

[0072] The answer information aggregation module is used to aggregate all the answer fragment information through the summary aggregation module to generate aggregated answer information;

[0073] The answer storage module is used to store the summary answer information and the corresponding user question information.

[0074] above Figure 2 The apparatus for enhanced generation of multi-document and multi-paragraph search queries in an embodiment of the present invention is described in detail from the perspective of modular functional entities, and the electronic device in an embodiment of the present invention is described in detail from the perspective of hardware processing.

[0075] Figure 3 7 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. The electronic device 700 may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 710 (for example, one or more processors) and a memory 720, and one or more storage media 730 (for example, one or more mass storage devices) storing application programs 733 or data 732. Among them, the memory 720 and the storage medium 730 can be temporary storage or permanent storage. The program stored in the storage medium 730 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the electronic device 700. Furthermore, the processor 710 can be configured to communicate with the storage medium 730 to execute a series of instruction operations in the storage medium 730 on the electronic device 700.

[0076] The electronic device 700 may also include one or more power supplies 740, one or more wired or wireless network interfaces 750, one or more input and output interfaces 750, and / or one or more operating systems 731, such as Windows Serve, Mac OS X, Unix, Linux, FreeBSD, etc. It will be appreciated by those skilled in the art that Figure 3 The structure of the electronic device shown does not constitute a limitation on the electronic device, and may include more or less components than shown in the figure, or combine some components, or arrange the components differently.

[0077] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are executed on a computer, the computer executes the steps of the method for enhanced generation of multi-document and multi-paragraph search queries.

[0078] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described system, device, or unit can refer to the corresponding process in the aforementioned method embodiment and will not be repeated here.

[0079] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or partly contributed to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0080] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for enhanced generation of multi-document multi-paragraph search queries, characterized in that: The method for enhanced generation of multi-document multi-paragraph search queries comprises: Receive user problem information; Decompose user question information into several sub-questions through the question decomposition model; Calling the text search module or the network search module for each sub-question in turn to generate answer fragment information respectively; All answer fragment information is summarized through the summary module to obtain the context information of the user's question information, and the summary answer information is generated based on the context information and all answer fragment information; The summary answer information and the corresponding user question information are stored.

2. The method for enhanced generation of multi-document multi-paragraph search queries according to claim 1, characterized in that: Also includes: Obtain data information, establish a database according to the field, the database includes several levels of indexes, and store the data information in the corresponding indexes.

3. The method for enhanced generation of multi-document multi-paragraph search queries according to claim 2, characterized in that: The disassembly process of the problem disassembly model includes: Reorganize user questions and break them down into several sub-questions based on technical fields and answer generation methods; The answer generation method includes: database text search generation and network search generation.

4. The method for enhanced generation of multi-document multi-paragraph search queries according to claim 3, characterized in that: The step of calling the text search module or the network search module for each sub-question in sequence to generate answer fragment information includes: For each sub-problem, determine whether the sub-problem can be decomposed through the problem decomposition model. If so, decompose the sub-problem into several subordinate questions and generate answer fragment information for each subordinate question. If not, directly generate answer fragment information.

5. The method for enhanced generation of multi-document multi-paragraph search queries according to claim 4, characterized in that: Generating answer fragment information for each subordinate question includes: Determine whether the answer generation method for the subordinate question is applicable to database text search generation or network search generation; if applicable to database text search generation, call the text search module to generate answer fragment information; if applicable to network search generation, call the network search module to generate answer fragment information.

6. The method for enhanced generation of multi-document multi-paragraph search queries according to claim 5, characterized in that: The directly generated answer fragment information includes: Determine whether the answer generation method of the sub-question is applicable to database text search generation or network search generation; if applicable to database text search generation, call the text search module to generate answer fragment information; if applicable to network search generation, call the network search module to generate answer fragment information.

7. A device for enhanced generation of multi-document multi-paragraph search queries, characterized in that: include: A question receiving module is used to receive user question information; The sub-question decomposition module is used to decompose the user question information into several sub-questions through the question decomposition model; An answer fragment generation module is used to call the text search module or the network search module to generate answer fragment information for each sub-question in turn; The answer information aggregation module is used to aggregate all the answer fragment information through the summary aggregation module to generate aggregated answer information; The answer storage module is used to store the summary answer information and the corresponding user question information.

8. An electronic device, comprising a memory and at least one processor, wherein instructions are stored in the memory; The at least one processor calls the instructions in the memory to enable the electronic device to execute the various steps of the method for enhanced generation of multi-document and multi-paragraph search queries as described in any one of claims 1 to 6.

9. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed by the processor, the various steps of the method for enhanced generation of multi-document and multi-paragraph search queries as claimed in any one of claims 1 to 6 are implemented.

Images