Document question-answering method and system based on multi-path recall and noise filtering optimization

Abstract

The invention provides a document question and answer method and system based on multi-path recall and noise filtering optimization, and belongs to the technical field of natural language processing. A multi-granularity document cutting and multi-path recall module, a document block processing and rearranging module, a context filling and global structure supplementing module, a filtering and picture information supplementing module based on an LLM model and a final answer generating module are adopted. According to the method, complete context information of a long document or complex query can be effectively obtained, high-efficiency duplicate removal is performed in a recall stage, high-correlation document blocks are screened out, occupation of redundant information on computing resources is reduced, recall efficiency is improved, global structure relevance among documents is fully utilized, and integrity and logicality of answer generation are ensured.

Description

Technical Field

[0001] The present invention belongs to the technical field of natural language processing, and in particular relates to a document question-answering method and system based on multi-path recall and noise filtering optimization. Background Art

[0002] With the rapid growth of information data, traditional document retrieval methods are constantly evolving, but they still mainly rely on keyword matching. The core of these methods is to match the user's query through a pre-built static index, and then return relevant content from the document library. Based on this type of method, existing document retrieval systems have developed more and more complete solutions, such as semantic retrieval technology, multi-granularity document segmentation, BM25-based optimized recall, or unstructured data processing technology, etc.

[0003] The above methods still have some problems: (1) Incomplete context understanding and recall. Current document retrieval is difficult to effectively capture complete context information when processing long documents or complex semantic queries. Keyword matching-based methods cannot cross document paragraph boundaries, resulting in the inability to recall all content that is highly relevant to the query semantics. This lack of context will affect the accuracy of answer generation to a certain extent. In some scenarios, existing technologies may repeatedly recall document blocks with the same content. Such redundant information will occupy computing resources and affect system efficiency. (2) High recall repetitiveness. There are situations where document blocks with the same content are repeatedly recalled. Such redundant information will occupy computing resources and affect system efficiency. (3) Noise information interference. It is easy to introduce a large number of irrelevant or low-relevance document blocks. This noise information not only increases the computational burden of subsequent document processing, but may also directly interfere with the reasoning results of the LLM model, resulting in deviations in the generated answers. (4) Lack of association of global structural information. It mainly focuses on local semantic relevance and ignores the global structural information between documents. For example, some answers may be located in the upper structure or other related parts of the document, but due to the lack of a global association mechanism, these key contents are not effectively utilized.

[0004] How to improve the efficiency and accuracy of document recall has always been a problem worth exploring in this field. Summary of the invention

[0005] In view of this, the present invention proposes a document question answering method and system based on multi-way recall and noise filtering optimization, which can effectively obtain the complete context information of long documents or complex queries, and efficiently deduplicate in the recall stage, screen out highly relevant document blocks, reduce the occupation of computing resources by redundant information, improve recall efficiency, make full use of the global structural correlation between documents, and ensure the integrity and logic of answer generation.

[0006] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0007] In a first aspect, the present invention proposes a document question answering method based on multi-way recall and noise filtering optimization, comprising:

[0008] (1) Preprocessing the document dataset and cutting it into multiple granularities to obtain long document blocks and short document blocks corresponding to each document, using a large language model to generate direct answers based on the question text, and using a multi-way recaller to generate candidate document blocks based on the direct answers and the question text, wherein the candidate document blocks include long document blocks and short document blocks;

[0009] (2) For the document blocks obtained by directly answering as the recall input, retain the document block corresponding to the file name with the highest frequency of occurrence; perform a deduplication operation on the retained document block and the remaining candidate document blocks to obtain a deduplicated long document block list and a short document block list, and further screen the candidate document blocks in the deduplicated list;

[0010] (3) filling in the context and supplementing the global structure of the candidate document blocks obtained in step (2) to obtain several candidate documents;

[0011] (4) Several candidate documents are input into the large language model respectively. The large language model generates a direct answer for each candidate document based on the question text, and filters the candidate documents or supplements the image information based on the answer results to obtain updated candidate documents;

[0012] (5) The updated candidate documents are input into the large language model, and the large language model is used to generate the final answer based on the question text.

[0013] Furthermore, the multi-way recaller includes a vector-based long document block recaller, a bm25-based long document block recaller, a vector-based short document block recaller, and a bm25-based short document block recaller; each recaller takes the direct answer and question text as input, and recalls a preset number of corresponding long document blocks and short document blocks as candidate document blocks.

[0014] Furthermore, after the document is cut into long document blocks and short document blocks, the image information in the document blocks is removed.

[0015] Furthermore, in step (2), for the four groups of document blocks obtained by directly answering as recall input, the top N document blocks in each group are selected respectively; the file names corresponding to the selected document blocks are extracted, the frequency of occurrence of each file name is counted, and the file name with the highest frequency of occurrence is selected; the document block with the file name with the highest frequency of occurrence is filtered out from the selected document blocks, and it is added to the candidate document block list.

[0016] Furthermore, in step (2), deduplication operations are performed based on the unique ID identifiers of the long document block and the short document block, respectively, and the topM document blocks are selected from the long document block list and the short document block list after deduplication, and the document blocks with relevance scores lower than the threshold are discarded; the relevance scores of the document blocks are calculated based on the reranker model.

[0017] Furthermore, the step (3) comprises:

[0018] (3-1) context filling is performed on the obtained document block to supplement relevant context information;

[0019] (3-2) Document blocks with the same file name are concatenated in the order in which they appear in the document to obtain several documents, and the overlapping parts are deleted; if there are unrecalled text segments between adjacent document blocks, the length of the unrecalled text segments is further determined. If the length of the unrecalled text segments is less than the length of any document block, the unrecalled text segments are added to the document block, otherwise the unrecalled text segments are ignored;

[0020] (3-3) If any two documents have the same parent document, the parent document will be recalled from the document list.

[0021] (3-4) Encode each document into a vector, calculate the similarity between each pair of documents, and remove the document with a low score according to the relevance score for a pair of documents whose similarity is greater than a threshold. The relevance score of the document is calculated according to the reranker model.

[0022] Furthermore, the filling strategy adopted in step (3-1) includes upward filling and downward filling. The filling length is preset during filling, and the filling is stopped when one of the following conditions is met:

[0023] a) The filling length has been reached and the filling portion does not contain an incomplete form;

[0024] b) The filling length has been reached and the current filling contains incomplete table information, in this case, the entire table will be filled and the process will stop;

[0025] c) If a title appears during the filling process, stop filling immediately.

[0026] Furthermore, the step (4) comprises:

[0027] Several candidate documents are input into the large language model respectively, and the large language model determines whether each candidate document contains the answer. If it contains the answer, the model outputs a direct answer for the candidate document; according to the number of candidate documents containing the answer, the similarity between the two direct answers is calculated, and the candidate documents corresponding to the pair of direct answers with similarity greater than the threshold are eliminated according to the relevance score. The relevance score of the document is calculated according to the reranker model; if only one candidate document contains the answer, there is no need to calculate the similarity;

[0028] If all candidate documents do not contain the answer, the image information in the candidate documents is restored and the candidate documents are updated.

[0029] Furthermore, the step (5) comprises:

[0030] (5-1) Re-rank the updated candidate documents according to the document relevance scores calculated by the last reranker model, placing documents with high scores at the beginning and end, and documents with low scores in the middle;

[0031] (5-2) All rearranged documents and question text are input into the large language model to obtain the final answer.

[0032] In a second aspect, the present invention proposes a document question answering system based on multi-way recall and noise filtering optimization, which is used to implement the above-mentioned document question answering method based on multi-way recall and noise filtering optimization.

[0033] The beneficial effects of the present invention are:

[0034] (1) The present invention improves the relevance of answers through a multi-way recall mechanism. Through multi-granularity document segmentation and multi-way recall strategies, the overall recall capability of documents containing answers is significantly improved, effectively making up for the shortcomings of existing technologies in terms of incomplete recall. The multi-way recall method of long and short document blocks combined with the BM25 algorithm and vectors can be efficiently applied to scenarios where answers are precisely located.

[0035] (2) The present invention enhances information integrity through context filling. The context filling strategy completes the context information as much as possible while ensuring a reasonable length, improves the logic and integrity of the recalled content, and makes up for the limitations of the existing technology on the boundaries of document paragraphs, making the generation of the final answer more comprehensive and accurate.

[0036] (3) The present invention improves resource utilization through deduplication and re-ranking mechanisms. In the recall phase, a deduplication mechanism is introduced to filter duplicate content through document block IDs, thereby avoiding the problem of large-scale computing resource occupation due to repeated recall of the same content in the prior art; the re-ranking phase further optimizes the ranking of candidate document blocks, so that system efficiency and resource utilization can be significantly improved.

[0037] (4) The present invention ensures information purity through noise filtering. The noise filtering technology based on the LLM model is introduced to screen the recalled documents, remove irrelevant or low-relevance document blocks, and perform vector deduplication on document blocks with similar output answers. This optimization significantly reduces the noise information in the recall results, ensures the information purity of the reference documents, and improves the accuracy of answer generation.

[0038] (5) The present invention enhances contextual relevance through a global structure supplementation mechanism. Through the global structure supplementation mechanism, the recalled contents that share the same parent document are merged, further enhancing the semantic relevance between documents and ensuring that the recalled contents contain sufficient global information, making the generated answers more complete and coherent. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0040] Figure 1 Schematic diagram of the framework of the document question answering method based on multi-way recall and noise filtering optimization. DETAILED DESCRIPTION

[0041] The present invention is further described and illustrated below in conjunction with specific embodiments. The embodiments are merely exemplary of the present disclosure and do not define the scope of limitation. The technical features of each embodiment of the present invention may be combined accordingly without conflicting with each other.

[0042] The accompanying drawings are only schematic illustrations of the present invention and are not necessarily drawn to scale. Some of the block diagrams shown in the accompanying drawings are functional entities that do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0043] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the steps. For example, some steps may be decomposed, while some steps may be combined or partially combined, so the actual execution order may change according to the actual situation.

[0044] like Figure 1As shown, the present invention proposes a document question answering method based on multi-way recall and noise filtering optimization, which includes a multi-granularity document cutting and multi-way recall module, a document block processing and rearrangement module, a context filling and global structure supplement module, an LLM model-based filtering and image information supplement module, and a final answer generation module. Each module is explained below.

[0045] (I) Multi-granularity document cutting and multi-channel recall module

[0046] For some situations where it is difficult to retrieve relevant information through the question text, the present invention generates a direct answer with the help of the background knowledge of the large language model (LLM). The answer to the direct answer may be the correct answer or may have a pattern similar to the correct answer, and both situations are helpful for subsequent related document retrieval.

[0047] For some questions that require a long context to answer, the information in small-grained document blocks is too fragmented and may not be able to recall the complete context or even the correct context, which ultimately leads to incomplete or wrong answers. For some questions where the answer only appears in one or two sentences, if only large document blocks are used, the similarity may be reduced due to the large amount of irrelevant information contained in the document blocks, resulting in failure to successfully recall.

[0048] For example, for the question text: "How many levels are rcp user permissions divided into, and which levels are they?"

[0049] The correct answer is: "ZXUN RCP can provide an independent user management system to manage user permissions. User permissions can be divided into three levels: super user permissions, operator permissions, and browsing user permissions. Users can create and manage according to their granted permissions."

[0050] It can be seen that the sentence can be correctly retrieved by adopting small-grained document blocks. Therefore, adopting small-grained document blocks can better solve the problem that the answers are concentrated in a very small range.

[0051] In the multi-granularity document cutting and multi-way recall module, a multi-way recall mechanism based on vector and BM25 algorithm is adopted to improve the overall recall rate. Among them, the BM25 algorithm uses keywords for recall, which can supplement the shortcomings of vector recall and improve the recall rate in the recall stage.

[0052] In a specific implementation of the present invention, the calculation process of the multi-granularity document cutting and multi-channel recall module includes:

[0053] (1-1) Document Processing

[0054] A multi-granularity document segmentation mechanism is adopted to segment documents with different file names into long document chunks (chunk_size=512) and short document chunks (chunk_size=180), and mark the hierarchical relationship between documents with different file names; each document chunk corresponds to the file name of the document to which it belongs, and the image information in the document chunk is deleted, because the image information has a lot of noise, and directly adding image information to the document may increase the difficulty of the LLM model to understand the document; among them, the chunk_size of the long document chunk is set to 512, because the maximum length of the vector model input is limited to 512; the chunk_size of the short document chunk is set to 180, because the maximum length of the hierarchical title in all documents is 136, that is, for this document, each document chunk can only contain text of (chunk_size-136) length. Therefore, setting the chunk_size of the short document chunk to 180 is a reasonable choice.

[0055] (1-2) Input the question text directly into the LLM model and let the LLM model give the answer directly to get a direct answer.

[0056] For example, the LLM model prompt word prompt used in this embodiment is as follows:

[0057] ""

[0058] You are an expert in computer operation and maintenance. You can perfectly use your knowledge to answer questions related to operation and maintenance. Please answer concisely.

[0059] Now there is a question about **{topic}**, the question is:

[0060] {query_str}

[0061] Your answer:

[0062] ""

[0063] In a specific scenario:

[0064] Question Text: "What protocol does AMF use to ensure the time is correct?"

[0065] Direct answer: "AMF (Adobe Media Framework) uses NTP (Network Time Protocol) to ensure the correct time. NTP is a network protocol used to synchronize computer clocks to ensure that operations such as audio and video streaming are performed at the correct time."

[0066] From the above examples, we can see that in the absence of any context, the answer generated by the LLM model is close to correct. At this time, using the direct answer to query, relevant documents can be retrieved more accurately.

[0067] (1-3) Two recall methods, one based on vector and the other based on BM25 algorithm, are adopted, and recallers corresponding to long document blocks and short document blocks are constructed at the same time, namely, the vector-based long document block recaller (long_embed_retriever) and the bm25-based long document block recaller (long_bm25_retriever), the bm25-based long document block recaller (short_embed_retriever) and the bm25-based short document block recaller (short_bm25_retriever), a total of four recallers.

[0068] In this embodiment, the vector-based recaller adopts the open source model puff-large-v1, which is based on a 24-layer BERT architecture and has an output vector dimension of 4096.

[0069] (1-4) The direct answer and question text are used as the input of the above four recallers respectively. Each recaller is defined to recall 20 most relevant document blocks each time. In this way, a total of 160 document blocks can be obtained, of which 80 are long document blocks and 80 are short document blocks.

[0070] (II) Document Block Processing and Rearrangement Module

[0071] In this module, by giving priority to retaining the key document blocks recalled by direct answers, the problem of key document loss due to insufficient relevance can be effectively reduced, ensuring that the recalled document list contains enough answer clues, thereby improving the accuracy of answer generation. And by rearranging long document blocks and short document blocks separately, the scoring bias caused by differences in document block lengths can be avoided, while retaining the advantages of multi-granularity recall. This processing method can not only ensure the information coverage of long document blocks, but also give play to the advantages of short document blocks in accurate answers, thereby improving the overall effect of answer generation.

[0072] In a specific implementation of the present invention, the calculation process of the document block processing and rearrangement module includes:

[0073] (2-1) For the 4×20=80 document blocks obtained by directly answering as recall input, each recaller selects the top 3 document blocks, and finally obtains 4×3=12 document blocks. Next, extract the file names corresponding to these 12 document blocks, count the frequency of each file name, and select the file name with the highest frequency. Then, filter out the document block with the file name with the highest frequency from these 12 document blocks and add it to the candidate document block list.

[0074] (2-2) In the vector-based recall and the BM25 algorithm-based recall, the same document block may be recalled repeatedly. Therefore, it is necessary to deduplicate based on the ID of the document block to avoid the subsequent reranker model from repeatedly calculating the same text content. It should be noted that since the IDs of long document blocks and short document blocks are inconsistent, it is necessary to deduplicate in the long document block list and the short document block list respectively to obtain the deduplicated long document block list and the short document block list.

[0075] (2-3) The long document block list and the short document block list after deduplication are respectively input into the reranker model to calculate the relevance score for re-ranking. For each list, the top 5 document blocks with the highest scores are selected, and the document blocks with scores lower than τ are discarded (τ is 0.2 in this embodiment). Document blocks with scores lower than τ are considered to have low relevance to the question text and are not involved in subsequent processing. In the end, the total number of retained document blocks does not exceed 10.

[0076] In this embodiment, the reranker model uses the open source bge-reranker-v2-m3, which is based on a 24-layer BERT architecture and is trained with a large amount of Chinese and English data. According to the different characteristics of long document blocks and short document blocks, the two are re-ranked separately. This is because the lengths of the two document blocks are inconsistent. Although the long document block contains more comprehensive answers, it is also easier to introduce noise information, which may lead to a low re-ranking score, which will be at a disadvantage when directly compared with the score of the short document block.

[0077] (III) Context filling and global structure supplementation module

[0078] In this module, the reranker model in the previous module reorders the recall results according to the relevance of the document block to the question text. However, some document blocks containing answers recalled through direct answers may be discarded in the reranking process due to their low relevance to the question text. To avoid the loss of key documents, these top-ranked document blocks recalled through direct answers are directly added to the recalled document list.

[0079] In addition, it is generally believed that text paragraphs separated by titles have higher knowledge cohesion, so when completing information, priority is given to completing until the title line; if no title is encountered, then the maximum length is completed. For table information, since the data in the same table is highly correlated, when completing table information, the maximum length limit can be exceeded to ensure the integrity of the entire table content.

[0080] The present invention ensures that the recalled document content has better contextual coherence and logic by completing the title and table information, and gives priority to completing the title and table integrity to ensure the integrity of the information, while avoiding the interference of irrelevant content, improving the model's ability to understand relevant information, and thus improving the accuracy of answer generation. By splicing document blocks belonging to the same document, the recalled document blocks are combined into a coherent document to improve the continuity of the document. Retrieve shared parent documents and identify the relevance between documents. Since parent documents usually have stronger globality, and the question text recalls mostly local information, the correct answer may sometimes be located in the parent document. Therefore, the completeness of the recall and the accuracy of the answer are improved by supplementing the parent document. Finally, through the document vector, the documents with too high similarity are deduplicated to prevent the subsequent LLM model from reading in multiple similar documents, resulting in a waste of computing resources.

[0081] In a specific implementation of the present invention, the calculation process of the context filling and global structure supplementation module includes:

[0082] (3-1) Perform context filling on the obtained document block to supplement relevant context information.

[0083] The specific filling strategy includes upward filling and downward filling. The filling length is preset during filling. In this embodiment, the filling length is set to 768. When one of the following conditions is met, the filling is stopped:

[0084] a) The filling length has been reached and the filling portion does not contain an incomplete form;

[0085] b) The filling length has been reached and the current filling contains incomplete table information. In this case, the filling will be stopped after the entire table is filled. Under this condition, the filling length is allowed to exceed the preset filling length;

[0086] c) If a title appears during the filling process, stop filling immediately;

[0087] (3-2) The document blocks with the same file name are spliced ​​together. The specific splicing strategy is as follows:

[0088] Sort document blocks belonging to the same document by their starting position in the document.

[0089] The sorted document blocks are spliced ​​in sequence. If there are overlapping parts between adjacent document blocks, the overlapping parts are deleted and the document blocks are spliced ​​into a new continuous document block. If there are unrecalled text segments between adjacent document blocks, the length of the unrecalled text segments is further determined. If the length of the unrecalled text segments is less than the length of any document block, the unrecalled text segments are added to the document blocks, and the three are spliced ​​into a new continuous document block. If the length of the unrecalled text segments is greater than the length of any document block, the unrecalled text segments are ignored, and the two document blocks are directly spliced ​​into a new discontinuous document block using "\n\n".

[0090] Eventually all document blocks belonging to the same file name are merged into a complete document.

[0091] (3-3) To avoid retrieval being trapped in local information, any two documents will be checked to see if they share the same parent document. If a common parent document is found, the parent document is added to the list of recalled documents, thus ensuring that the recalled content contains sufficient global information.

[0092] For example, for the question text: "What data do you generally need to configure when PCF and NRF are connected?"

[0093] Assume that the recalled documents include "rcp - Initial Configuration (Manual Deployment) - Configure NRF Docking Data - Configure HTTP" and "rcp - Initial Configuration (Manual Deployment) - Configure NRF Docking Data - Configure NRF Server".

[0094] These two documents share a common parent document "rcp - Initial Configuration (Manual Deployment) - Configure NRF Docking Data". At this time, the parent document will be added to the recalled document list. The parent document supplements the structured information between the documents and contains a more general answer.

[0095] (3-4) The relevance score of each document in the recalled document list is calculated based on the reranker model. Documents with high scores have stronger relevance, while documents with low scores have weaker relevance.

[0096] (3-5) Encode the document using the vector model puff-large-v1 to generate a document vector. Using the document vector, calculate the cosine similarity between each pair of documents. If the similarity between two documents exceeds the set threshold, the document with the lower score is removed based on the relevance score.

[0097] (IV) Filtering and image information supplementation module based on LLM model

[0098] Considering that the recalled documents contain a certain amount of noise, there will be documents with low relevance in the document list, and there will also be documents with similar answers (which may be correct answers or incorrect answers), but the documents are not similar as a whole. Therefore, this module uses the LLM model filtering method to process each document in parallel, filter or deduplicate documents without answers or with similar answers, and ensure the information purity of the reference documents input into the LLM model in the last step. In addition, for questions that the LLM model cannot answer, the answers are likely to exist in the pictures of the original documents. Therefore, it is necessary to restore the picture information to improve the accuracy of the answer.

[0099] In a specific implementation of the present invention, the calculation process of the filtering and picture information supplementation module based on the LLM model includes:

[0100] (4-1) Input multiple documents obtained in the previous module into the LLM model respectively, and determine whether each document contains the answer by asking the model. If the document contains the answer, the model will output a concise and direct answer. This step uses parallel processing to save computing time. Through the judgment of the LLM model, documents that do not contain answers can be eliminated; at the same time, for the answers output by the LLM model, the similarity between the two answers is judged. If the similarity exceeds the set threshold, the documents with lower scores are eliminated according to the document relevance score calculated by the last reranker model, so as to achieve the purpose of deduplication of documents that output similar answers.

[0101] (4-2) If the LLM model determines that all documents do not contain the answer, it is inferred that the answer may exist in the picture. At this time, it is necessary to restore the picture information of the multiple documents obtained in the previous module; when restoring the picture information, it is necessary to parse the picture first, and fill the text obtained from the analysis into the corresponding document. The picture analysis can be achieved through existing technologies and will not be repeated here. Due to the diverse structure of the picture, some of the analysis results may contain more noise, and directly adding all of them to the original document will increase the noise of the document information. Therefore, the present invention will only look for the answer in the picture information of the corresponding document when the model has not found the answer in all retrieved documents.

[0102] (V) Final answer generation module

[0103] This module adjusts the position of the document in the context, guides the model to focus on more relevant content, and improves the accuracy of the answer. At the same time, it avoids the problem of losing key documents due to card threshold filtering, and enhances the flexibility and rationality of the system.

[0104] In a specific implementation of the present invention, the calculation process of the final answer generation module includes:

[0105] (5-1) Re-rank the documents filtered by the previous module according to the document relevance scores calculated by the last reranker model, placing documents with high scores at the head and tail, and documents with low scores in the middle. This is because when a large model processes a long input, it will pay more attention to the head and tail of the input, and relatively critical documents will be placed at the head and tail of the input to enhance the model's attention to important information.

[0106] (5-2) All the recalled documents and the question text after rearrangement are input into the LLM model to obtain the final answer. Considering that the document containing the answer is usually ranked in the top 1, but in some cases it may be located at the back, in step (5-1), instead of setting a threshold to directly filter out other documents, more candidate documents are retained for comprehensive processing in this step.

[0107] The above modules can be used to construct a document question-answering system with multi-channel recall and noise filtering optimization, which can be applied to any device with data processing capabilities, such as a computer or other device. The system embodiment can be implemented by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by the processor of any device with data processing capabilities in which it is located to read the corresponding computer program instructions in the non-volatile memory into the memory and run them.

[0108] The above-described embodiment only expresses an optional implementation mode of the present invention, and its description is relatively specific and detailed, but it cannot be understood as limiting the scope of the present invention. For ordinary technicians in this field, several modifications and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention.

Claims

1. A document question answering method based on multi-way recall and noise filtering optimization, characterized in that: include: (1) Preprocessing the document dataset and cutting it into multiple granularities to obtain long document blocks and short document blocks corresponding to each document, using a large language model to generate direct answers based on the question text, and using a multi-way recaller to generate candidate document blocks based on the direct answers and the question text, wherein the candidate document blocks include long document blocks and short document blocks; (2) For the document blocks obtained by directly answering as the recall input, retain the document block corresponding to the file name with the highest frequency of occurrence; perform a deduplication operation on the retained document block and the remaining candidate document blocks to obtain a deduplicated long document block list and a short document block list, and further screen the candidate document blocks in the deduplicated list; (3) filling in the context and supplementing the global structure of the candidate document blocks obtained in step (2) to obtain several candidate documents; (4) Several candidate documents are input into the large language model respectively. The large language model generates a direct answer for each candidate document based on the question text, and filters the candidate documents or supplements the image information based on the answer results to obtain updated candidate documents; (5) The updated candidate documents are input into the large language model, and the large language model is used to generate the final answer based on the question text.

2. The document question answering method based on multi-channel recall and noise filtering optimization according to claim 1 is characterized in that: The multi-way recaller includes a vector-based long document block recaller, a bm25-based long document block recaller, a vector-based short document block recaller, and a bm25-based short document block recaller; each recaller takes the direct answer and question text as input, and recalls a preset number of corresponding long document blocks and short document blocks as candidate document blocks.

3. The document question answering method based on multi-channel recall and noise filtering optimization according to claim 1 is characterized in that: After the document is cut into long document blocks and short document blocks, the image information in the document blocks is removed.

4. The document question answering method based on multi-channel recall and noise filtering optimization according to claim 1 is characterized in that: In step (2), for the four groups of document blocks obtained by directly answering as recall input, select the topN document blocks in each group respectively; extract the file names corresponding to the selected document blocks, count the frequency of occurrence of each file name, and select the file name with the highest frequency; filter out the document block with the file name with the highest frequency from the selected document blocks, and add it to the candidate document block list.

5. The document question answering method based on multi-channel recall and noise filtering optimization according to claim 1 is characterized in that: In step (2), deduplication is performed based on the unique ID identifiers of the long document block and the short document block, and the topM document blocks are selected from the long document block list and the short document block list after deduplication, and the document blocks with relevance scores lower than the threshold are discarded; the relevance scores of the document blocks are calculated based on the reranker model.

6. The document question answering method based on multi-channel recall and noise filtering optimization according to claim 1 is characterized in that: The step (3) comprises: (3-1) context filling is performed on the obtained document block to supplement relevant context information; (3-2) Document blocks with the same file name are concatenated in the order in which they appear in the document to obtain several documents, and the overlapping parts are deleted; if there are unrecalled text segments between adjacent document blocks, the length of the unrecalled text segments is further determined. If the length of the unrecalled text segments is less than the length of any document block, the unrecalled text segments are added to the document block, otherwise the unrecalled text segments are ignored; (3-3) If any two documents have the same parent document, the parent document will be recalled from the document list. (3-4) Encode each document into a vector, calculate the similarity between each pair of documents, and remove the document with a low score according to the relevance score for a pair of documents whose similarity is greater than a threshold. The relevance score of the document is calculated according to the reranker model.

7. The document question answering method based on multi-channel recall and noise filtering optimization according to claim 1 is characterized in that: The filling strategy adopted in step (3-1) includes upward filling and downward filling. The filling length is preset during filling, and the filling is stopped when one of the following conditions is met: a) The filling length has been reached and the filling part does not contain an incomplete form; b) The filling length has been reached and the current filling contains incomplete table information, in this case, the entire table will be filled and the process will stop; c) If a title appears during the filling process, stop filling immediately.

8. The document question answering method based on multi-channel recall and noise filtering optimization according to claim 1 is characterized in that: The step (4) comprises: Several candidate documents are input into the large language model respectively, and the large language model determines whether each candidate document contains the answer. If it contains the answer, the model outputs a direct answer for the candidate document; according to the number of candidate documents containing the answer, the similarity between the two direct answers is calculated, and the candidate documents corresponding to the pair of direct answers with similarity greater than the threshold are eliminated according to the relevance score. The relevance score of the document is calculated according to the reranker model; if only one candidate document contains the answer, there is no need to calculate the similarity; If all candidate documents do not contain the answer, the image information in the candidate documents is restored and the candidate documents are updated.

9. The document question answering method based on multi-channel recall and noise filtering optimization according to claim 1, characterized in that: The step (5) comprises: (5-1) Re-rank the updated candidate documents according to the document relevance scores calculated by the last reranker model, placing documents with high scores at the beginning and end, and documents with low scores in the middle; (5-2) All rearranged documents and question text are input into the large language model to obtain the final answer.

10. A document question answering system based on multi-channel recall and noise filtering optimization, characterized in that: include: A multi-granularity document segmentation and multi-way recall module is used to pre-process and segment the document data set into multi-granularity to obtain long document blocks and short document blocks corresponding to each document, generate direct answers based on the question text using a large language model, and generate candidate document blocks based on the direct answers and the question text using a multi-way recaller, wherein the candidate document blocks include long document blocks and short document blocks; The document block processing and rearrangement module is used to retain the document block corresponding to the file name with the highest frequency of occurrence for the document blocks obtained by directly answering as the recall input; perform deduplication operation on the retained document block and the remaining candidate document blocks to obtain a deduplicated long document block list and a short document block list, and further screen the candidate document blocks in the deduplicated list; A context filling and global structure supplementation module is used to fill in the context and supplement the global structure of the candidate document block to obtain a number of candidate documents; The filtering and image information supplementation module based on the LLM model is used to input several candidate documents into the large language model respectively. The large language model generates a direct answer for each candidate document according to the question text, and filters the candidate documents or supplements the image information according to the answer result to obtain the updated candidate document; The final answer generation module is used to input the updated candidate documents into the large language model and use the large language model to generate the final answer based on the question text.

Images