Multi-document question and answer method and device based on multi-head self-attention and hierarchical enhancement

Abstract

The invention discloses a multi-document question and answer method and equipment based on multi-head self-attention and hierarchical enhancement, belongs to the technical field of natural language understanding, and aims to solve the problems that most of existing RAG solutions focus on document retrieval of single query and neglect the requirements of multi-aspect query, and the query efficiency is low. And the application of the RAG method in the multi-document question and answer task is limited. The method comprises the following steps: extracting hierarchical structure metadata from a document in a preset format, constructing a hierarchical document tree based on a hierarchical relationship, and obtaining a context relationship in the document in the preset format; determining one or more target retrieval paths based on the retrieval information; performing multi-document retrieval on the document under each target retrieval path through a multi-head self-attention embedding module to obtain a plurality of related document paragraphs; screening a target document paragraph from the related document paragraphs; and based on the paragraphs of the target document and the corresponding context relationships, sorting out question and answer results of the retrieval information in a large language model.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language understanding, and in particular to a multi-document question-answering method and device based on multi-head self-attention and hierarchical enhancement. Background Art

[0002] In the field of natural language processing, especially in document question answering (DQA) tasks, the Retrieval-Augmented Generation (RAG) framework has made significant progress in recent years, especially in single-document question answering tasks. However, with the increase in the number of documents, especially in multi-document question answering (MDQA) scenarios, traditional RAG methods such as ChatPDF began to encounter performance bottlenecks and the response accuracy dropped significantly. This phenomenon is attributed to multiple factors: first, when dealing with documents with similar content and structure, existing RAG methods often find it difficult to effectively distinguish and extract the most relevant information; second, user queries often contain complex meta-information, which not only increases the difficulty of retrieval, but also increases the complexity of generating responses, further reducing the accuracy of retrieval.

[0003] Specifically, the traditional RAG framework recalls document fragments related to the query through vector retrieval and ranking, and uses a large language model to generate responses through prompt technology. However, there is a significant semantic gap in this process: large language models are unable to fully utilize the precise information provided by the retriever, resulting in deviations between the generated content and the expected goal. In addition, while large language models provide powerful contextual learning capabilities, they are also accompanied by high training costs, data leakage risks, and generation hallucinations, which are particularly prominent in multi-document question-answering scenarios.

[0004] To overcome these challenges, researchers have proposed retrieval-augmented generation methods, which aim to improve the generation capability and credibility of large language models by integrating retrieval systems. However, most existing RAG solutions focus on document retrieval for a single query, ignoring the need for multi-faceted queries, which requires the simultaneous retrieval and processing of multiple documents with different contents. This limitation is particularly evident when dealing with complex, multi-dimensional queries, limiting the widespread application of RAG methods in multi-document question answering tasks. Summary of the invention

[0005] The embodiments of the present invention provide a multi-document question answering method and device based on multi-head self-attention and hierarchical enhancement, which are used to solve the following technical problems: most of the existing RAG solutions focus on document retrieval of a single query, ignoring the needs of multi-faceted queries, limiting the application of RAG methods in multi-document question answering tasks.

[0006] The embodiment of the present invention adopts the following technical solution: On the one hand, an embodiment of the present invention provides a multi-document question answering method based on multi-head self-attention and hierarchical enhancement, the method comprising: parsing document data in a document library and formatting it to obtain a preset format document; Extracting hierarchical metadata from the preset format document, and constructing a hierarchical document tree based on the hierarchical relationship between the hierarchical metadata, to obtain the contextual relationship in the preset format document; Acquire search information input by a user, and determine one or more target search paths based on the search information; Through the multi-head self-attention embedding module, multi-document retrieval is performed on the documents under each target retrieval path to obtain several relevant document paragraphs; Calculating the comprehensive matching degree between the relevant document paragraphs and the search information, and screening the target document paragraphs according to the comprehensive matching degree; Based on the target document paragraphs and the corresponding contextual relationships, the question-answering results of the retrieval information are sorted out in a large language model.

[0007] In a feasible implementation, the document data in the document library is parsed and formatted to obtain a document in a preset format, specifically including: Importing document data stored in a document library; wherein the document data at least includes plain text data, Word document data, PDF document data, image data, and table data; Based on a large language model and natural language processing technology, the document data is converted into a document in a preset format.

[0008] In a feasible implementation, based on a large language model and natural language processing technology, the document data is converted into a document in a preset format, specifically including: Through the large language model, plain text data, Word document data, and PDF document data are converted into Markdown format documents; For the image data in the PDF document data, use the PDFImageSearcher tool to extract the bitmap and SVG image of the image data, and generate an image description in combination with surrounding text, and generate a descriptive title for the image data based on a visual language model, and generate a Markdown format document based on the descriptive title and the image description; Extracting table titles, row labels, and column labels from the table data through a large language model, generating a semantic representation of the table, and generating a corresponding Markdown format document based on the semantic representation; The generated Markdown format documents are processed for readability through natural language processing technology.

[0009] In a feasible implementation, extracting hierarchical metadata from the preset format document, and constructing a hierarchical document tree based on the hierarchical relationship between the hierarchical metadata to obtain the contextual relationship in the preset format document specifically includes: Extracting hierarchical metadata from each preset format document; wherein the hierarchical metadata at least includes various structural elements of the document, and the structural elements at least include chapter titles, chapter subtitles, and paragraphs; Based on the hierarchical relationship between each structural element, a unique identifier is assigned to each structural element; wherein the identifier can be reflected in the hierarchical relationship between other structural elements; According to the identifier, each structural element is constructed into a hierarchical document tree; the root node of the hierarchical document tree is the title of the entire document, and the leaf nodes are each paragraph; The hierarchical document tree is traversed, and based on the connection relationship between each node, the context relationship in the preset format document is added.

[0010] In a feasible implementation, traversing the hierarchical document tree and adding a contextual relationship in the preset format document based on the connection relationship between each node specifically includes: Traversing the hierarchical document tree by a depth-first search algorithm, and recording the access order and access path of each node during the traversal process to obtain the contextual relationship between the nodes; The contextual relationship between the nodes is marked in different forms in the data of the preset format document; wherein, contextual hierarchical tags are added to text paragraphs; descriptive text is generated for table data and image data, and associated with related text paragraphs; When the content of the preset format document is updated, the preset format document is re-parsed and the hierarchical document tree is updated.

[0011] In a feasible implementation manner, based on the search information, determining one or more target search paths specifically includes: Based on different retrieval algorithms, multiple corresponding retrieval paths are constructed to obtain multiple retrieval paths; wherein the different retrieval algorithms at least include: TF-IDF algorithm, BM25 algorithm and vector similarity measurement algorithm; The retrieval information is subjected to path decision by a pre-trained decision router, and one or more target retrieval paths are determined from the multiple retrieval paths for document retrieval.

[0012] In a feasible implementation, a multi-head self-attention embedding module is used to perform multi-document retrieval on the documents under each target retrieval path to obtain several relevant document paragraphs, including: Obtaining, by means of the multi-head self-attention embedding module, a retrieval information embedding representation of the retrieval information and a document embedding representation of the document in the preset format; Adding the search information embedding representation to the front of the document embedding representation as an external knowledge anchor point of the document embedding representation to form an embedding combination; The embedding combination is input into a large language model, and a multi-document search is performed according to each target search path to obtain a number of related document paragraphs.

[0013] In a feasible implementation, obtaining the retrieval information embedding representation of the retrieval information and the document embedding representation of the preset format document through the multi-head self-attention embedding module specifically includes: The search information and the preset format documents in the document library are represented as text blocks containing a number of tokens, and are mapped to a vector space of fixed dimension through an embedding layer to obtain a corresponding initial embedding representation; The initial embedding representation is divided into a plurality of header data, and attention calculation is performed independently in each header data to obtain a corresponding retrieval information embedding representation and a document embedding representation; The attention calculation includes at least: linear transformation to generate query, key and value vectors, using dot product attention mechanism to calculate attention weight, weighted summation of value vectors, and concatenating the outputs of all head data and obtaining the final multi-head attention output through linear transformation.

[0014] In a feasible implementation manner, calculating the comprehensive matching degree between the relevant document paragraphs and the search information, and screening the target document paragraphs according to the comprehensive matching degree specifically includes: Converting the retrieved relevant document paragraphs and the retrieval information into vector representations; Based on the vector representation, respectively calculating a relevance score, a previous example similarity score, a neighbor similarity score, and a first example similarity score of each relevant document paragraph and the retrieval information; Performing weighted calculation on the relevance score, the previous case similarity score, the neighbor similarity score, and the first case similarity score to obtain a comprehensive matching degree of each relevant document paragraph; The relevant document paragraphs whose comprehensive matching degree is greater than a preset threshold are determined as the target document paragraphs.

[0015] On the other hand, an embodiment of the present invention further provides a multi-document question answering device based on multi-head self-attention and hierarchical enhancement, the device comprising: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor so that the at least one processor can execute the multi-document question answering method based on multi-head self-attention and hierarchical enhancement.

[0016] Compared with the prior art, the multi-document question answering method and device based on multi-head self-attention and hierarchical enhancement provided by the embodiment of the present invention has the following beneficial effects: The present invention creates a multi-document question-answering method based on a multi-head self-attention mechanism and hierarchical enhancement technology. The core of this method is to use the activation of the multi-head self-attention layer of the Transformer decoder to generate an embedded representation of the vector, rather than being limited to the feedforward layer. Different attention heads can capture different aspects of the data, thereby more comprehensively representing various aspects of the document data and retrieval information. This mechanism can more effectively handle complex multi-faceted queries and improve the accuracy of retrieval and generation. And by aligning the semantic gap between the large language model and the retriever, the ability to answer multiple documents is improved.

[0017] This paper introduces a multi-attention head mechanism and a hierarchical enhancement strategy, which not only improves the accuracy of retrieval and generation, but also enhances the model's ability to handle complex queries, providing a new solution for multi-document question answering tasks. This method aims to solve the problem of retrieval enhancement generation in multi-document question answering by improving the RAG framework, especially the challenges of dealing with multi-faceted queries and similar documents. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings: Figure 1 A flow chart of a multi-document question answering method based on multi-head self-attention and hierarchical enhancement provided by an embodiment of the present invention; Figure 2 A schematic diagram of the structure of a multi-document question-answering device based on multi-head self-attention and hierarchical enhancement provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0019] In order to enable those skilled in the art to better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0020] The embodiment of the present invention provides a multi-document question answering method based on multi-head self-attention and hierarchical enhancement, such as Figure 1 As shown, the multi-document question answering method based on multi-head self-attention and hierarchical enhancement specifically includes steps S101-S106: S101, parsing document data in a document library and formatting the data to obtain a document in a preset format.

[0021] Specifically, document data stored in a document library is imported; wherein the document data at least includes plain text data, Word document data, PDF document data, image data, and table data.

[0022] Furthermore, based on the large language model and natural language processing technology, the document data is converted into a document in a preset format, specifically including the following conversion methods: Through the large language model, plain text data, Word document data, and PDF document data are all converted into Markdown format documents.

[0023] For image data in PDF document data, the PDFImageSearcher tool is used to extract the bitmap and SVG images of the image data, and generate image descriptions in combination with the surrounding text. At the same time, a descriptive title is generated for the image data based on the visual language model, and a Markdown format document is generated based on the descriptive title and image description.

[0024] Through the large language model, the table title, row label and column label in the table data are extracted to generate the semantic representation of the table, and the corresponding Markdown format document is generated based on the semantic representation.

[0025] Finally, the generated Markdown format document is processed for readability through natural language processing technology.

[0026] As a feasible implementation, for PDF documents, a large language model is used to convert them into Markdown format, treating chapters as first-level headings and adding digital identifiers. The specific conversion process can be expressed as: DM(t)=Mc(DI(t),DI(t−1),DM(t−1)), where DI(t) represents the PDF document input at time t, DI(t-1) represents the PDF document input at time t-1, DM(t) represents the output Markdown document, DM(t−1) represents the Markdown document output at time t-1, and Mc is the large language model.

[0027] For images in PDF documents, PDFImageSearcher is used to extract bitmaps and SVG images, and images are retrieved through the API. The image description is generated by combining the surrounding text and the visual language model, and the visual language model is used to create a descriptive title for the image. The image description generation process can be expressed as: Dimg=I(I,T,V), where Dimg is the image description, I is the image, T is the text around the image, and V is the visual language model.

[0028] Table processing focuses on semantic elements, omitting data fields to reduce noise but retaining context. Table semantic extraction can be expressed as: Stable = T(T, H, R, C), where Stable is the semantic representation of the table, T is the table title, H is the row label, R is the column label, and C is the paragraph label.

[0029] Finally, through natural language processing (NLP) technologies such as word segmentation, entity recognition, and semantic analysis, the accuracy and readability of the converted document content, image description, and table semantic representation are ensured. NLP processing can be expressed as: Ptext=N(W,E,S), where Ptext is the processed text, W is the original text, E is the entity recognition result, and S is the semantic analysis result.

[0030] S102: extracting hierarchical metadata from a preset format document, and constructing a hierarchical document tree based on the hierarchical relationship between the hierarchical metadata to obtain a contextual relationship in the preset format document.

[0031] Specifically, hierarchical metadata is extracted from each preset format document; wherein the hierarchical metadata at least includes various structural elements of the document, and the structural elements at least include chapter titles, chapter subtitles, and paragraphs.

[0032] Furthermore, based on the hierarchical relationship between each structural element, a unique identifier is assigned to each structural element; wherein the identifier can reflect the hierarchical relationship between other structural elements. According to the identifier, each structural element is constructed into a hierarchical document tree; the root node of the hierarchical document tree is the title of the entire document, and the leaf nodes are each paragraph.

[0033] As a feasible implementation method, first, hierarchical metadata is extracted from the Markdown format document. These metadata include structural elements such as titles, subtitles, paragraphs, etc. of each chapter. Each structural element will be assigned a unique identifier, and the hierarchical relationship between them will also be recorded. For example, if the identifier of the title is 1.1, then the identifier of the subtitle under the title is 1.1.1, and so on. Next, a hierarchical document tree (Chapter Tree) is constructed based on these extracted hierarchical metadata. The root node of this document tree usually represents the title or overview of the entire document, each node represents a chapter or subchapter, and the child nodes of the node represent the subchapter or paragraph under the chapter. These nodes are connected to each other through hierarchical relationships, forming a complete tree structure together.

[0034] Furthermore, the hierarchical document tree is traversed, and based on the connection relationship between each node, the contextual relationship in the preset format document is added.

[0035] As a feasible implementation method, the hierarchical document tree is traversed by a depth-first search algorithm, and during the traversal process, the access order and access path of each node are recorded to obtain the contextual relationship between the nodes. The contextual relationship between the nodes is then marked in different forms in the data of the preset format document; wherein, contextual hierarchical tags are added to the text paragraphs; descriptive text is generated for the table data and the image data, and is associated with the relevant text paragraphs. When the content of the preset format document is updated, the preset format document is re-parsed and the hierarchical document tree is updated.

[0036] S103: Acquire search information input by the user, and determine one or more target search paths based on the search information.

[0037] Specifically, based on different retrieval algorithms, multiple corresponding retrieval paths are constructed to obtain multiple retrieval paths; wherein the different retrieval algorithms at least include: TF-IDF algorithm, BM25 algorithm and vector similarity measurement algorithm.

[0038] Furthermore, a pre-trained decision router is used to perform path decision on the retrieval information, and one or more target retrieval paths are determined from multiple retrieval paths for document retrieval.

[0039] As a feasible implementation, the multi-path retrieval module adopts an advanced multi-path retrieval strategy, which aims to accurately locate the most appropriate document paragraphs by exploring diverse retrieval paths in parallel and efficiently. These paths are constructed based on different retrieval algorithms and features, including but not limited to the classic TF-IDF (Term Frequency-InverseDocument Frequency) algorithm, BM25 (Best Matching 25) algorithm, and modern vector similarity measurement methods. The combined use of these methods greatly enriches the dimension and depth of retrieval.

[0040] This module also includes a feedback mechanism to adjust the retrieval path according to the model's prediction results to optimize future retrieval performance. Specifically, a decision router is trained to determine which retrieval path to select to obtain the best document paragraph based on the retrieval information. The router is a multi-classification task, and one or more retrieval paths can be selected for each retrieval information. The training of the decision router can be expressed as an optimization problem, where the objective function is to maximize the accuracy of the retrieval path selection. The objective function can be expressed as: ;in, are model parameters, x i To retrieve information, y i is the corresponding optimal search path, P( y i ∣ x i ; ) is given the retrieval information x i and model parameters Select the path in the case y i In this way, the decision router can dynamically adjust the retrieval path to adapt to the changing data distribution and retrieval requirements.

[0041] S104: Perform multi-document retrieval on the documents under each target retrieval path through a multi-head self-attention embedding module to obtain several related document paragraphs.

[0042] Specifically, through a multi-head self-attention embedding module, a retrieval information embedding representation of the retrieval information and a document embedding representation of a preset format document are obtained.

[0043] The retrieval information embedding representation is added to the front of the document embedding representation as an external knowledge anchor of the document embedding representation to form an embedding combination.

[0044] The embedding combination is input into the large language model, and multi-document retrieval is performed according to each target retrieval path to obtain several relevant document paragraphs.

[0045] As a feasible implementation, the goal of semantic alignment training for a large language model is to map the retrieved information to the same dimensional space as the token embedding of the large language model. This is achieved through a projection layer P: , where Eretrieval is the original embedding of the retrieved information and Eprojected is the projected embedding. The embedding of the retrieved information after projection has the same dimension as the token embedding of the large language model. The tokenizer of the large language model tokenizes the query Q and document D into token sequences and converts these tokens into embedding representations EQ and ED. In order to enhance the large language model's ability to understand the retrieved information, a multi-head attention embedding module A is introduced: ;in, { E D1 ,E D2 ,…,E Dm} represents the embedding set of retrieval information, and MultiHead represents the multi-head attention mechanism, which allows a large language model to simultaneously focus on the embeddings of multiple related information when processing retrieval information.

[0046] As a feasible implementation, in order to analyze the document in detail, the retrieval information embedding is added to the front of the document embedding as an external knowledge anchor to guide the large language model to focus on the key information in the document. In order to embed the retrieval information into the document embedding, the vector representation of the document and the retrieval information is generated through the multi-head attention embedding module. Let D be the document embedding representation, R be the retrieval information embedding representation, and embed R in front of D as an external knowledge anchor. This can be achieved by the following formula: ; Where E is the merged input sequence and Transformer represents the processing of the self-attention mechanism.

[0047] Furthermore, through the multi-head self-attention embedding module, the retrieval information embedding representation of the retrieval information and the document embedding representation of the preset format document are obtained, specifically including: The retrieval information and the pre-formatted documents in the document library are represented as text blocks containing several tokens, and mapped to a vector space of fixed dimension through an embedding layer to obtain the corresponding initial embedding representation. The initial embedding representation is divided into multiple header data, and attention calculation is performed independently in each header data to obtain the corresponding retrieval information embedding representation and document embedding representation; wherein the attention calculation at least includes: linear transformation to generate query, key and value vectors, using the dot product attention mechanism to calculate the attention weight, weighted summation of the value vector, and concatenation of the outputs of all header data and obtaining the final multi-head attention output through linear transformation.

[0048] As a feasible implementation, the present invention introduces a multi-head self-attention mechanism (Multi-HeadAttention), which is derived from the Transformer architecture and allows the model to pay attention to information at different positions in parallel when processing an input sequence. The module first represents a given document or retrieval information as a text block containing n tokens, and maps it to a vector space of fixed dimension through an embedding layer to form an initial embedding representation. Subsequently, the initial embedding representation is divided into multiple heads (assuming H), and attention calculations are performed independently in each head, including linear transformation to generate query, key and value vectors, using the dot product attention mechanism to calculate the attention weights, weighted summation of the value vector, and splicing the outputs of all heads and obtaining the final multi-head attention output through linear transformation. In order to adapt to documents of different lengths and complexities, the module is also optimized, such as adding position encoding to capture the positional relationship between tokens, using layer normalization and residual connections to stabilize the training process, and introducing a learnable scaling factor to prevent gradient problems. After these processes, the embedding module of the multi-attention head can generate an embedding representation that more comprehensively characterizes multi-source documents and query semantic information, providing strong support for processing complex and multi-faceted queries.

[0049] S105: Calculate the comprehensive matching degree between the relevant document paragraphs and the search information, and select the target document paragraphs according to the comprehensive matching degree.

[0050] Specifically, the retrieved relevant document paragraphs and the retrieval information are converted into vector representations. Based on the vector representations, the relevance score, the previous similarity score, the neighbor similarity score and the first similarity score of each relevant document paragraph and the retrieval information are calculated respectively.

[0051] Furthermore, the relevance score, the previous similarity score, the neighbor similarity score and the first similarity score are weighted to obtain the comprehensive matching degree of each relevant document paragraph. The relevant document paragraph with a comprehensive matching degree greater than a preset threshold is determined as the target document paragraph.

[0052] As a feasible implementation, the relevance score Srelevance is: ; Where Q represents the query vector, D represents the retrieved document vector, and cosine_similarity represents the cosine similarity, which is used to measure the proximity between the two in the semantic space.

[0053] Previous similarity score Sprecedent: ;in,{ D 1 ,D 2 ,…,D n} represents the collection of previously processed documents, and similarity can be Jaccard similarity or other metrics to capture the similarity between new documents and historical documents.

[0054] Neighbor similarity score Sneighbor: ;in, represents the i-th neighbor of the document vector D, k is the number of neighbors, and the score reflects the average similarity between the document and its neighbors.

[0055] The first similarity score Sfirst: ; Where D1 is the first document retrieved, and weight is a preset weight used to emphasize the relevance of the first document.

[0056] Finally, these similarity scores are combined by weighted summation to form a comprehensive matching degree S: Among them, w1, w2, w3 and w4 are the weights of the corresponding scores, which are determined through the optimization process to maximize the performance of the question answering system.

[0057] S106. Based on the target document paragraphs and the corresponding contextual relationships, the question-answering results of the retrieved information are sorted out in the large language model.

[0058] Specifically, through the large language model, the multiple retrieved target document paragraphs are integrated and analyzed, and paragraphs with contextual relationships with the target document paragraphs are obtained to assist in understanding, thereby generating question and answer results for the user's search information and outputting them to the question and answer interface for display to the user.

[0059] In addition, the embodiment of the present invention also provides a multi-document question answering device based on multi-head self-attention and hierarchical enhancement, such as Figure 2 As shown in FIG. 1 , the multi-document question answering device based on multi-head self-attention and hierarchical enhancement specifically includes: at least one processor; and a memory in communication with the at least one processor; wherein, The memory stores instructions executable by at least one processor to enable the at least one processor to perform: Parse the document data in the document library and format it to obtain a document in a preset format; Extracting hierarchical metadata from the preset format document, and constructing a hierarchical document tree based on the hierarchical relationship between the hierarchical metadata, to obtain the contextual relationship in the preset format document; Acquire search information input by a user, and determine one or more target search paths based on the search information; Through the multi-head self-attention embedding module, multi-document retrieval is performed on the documents under each target retrieval path to obtain several relevant document paragraphs; Calculating the comprehensive matching degree between the relevant document paragraphs and the search information, and screening the target document paragraphs according to the comprehensive matching degree; Based on the target document paragraphs and the corresponding contextual relationships, the question-answering results of the retrieval information are sorted out in a large language model.

[0060] Each embodiment of the present invention is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, equipment, and non-volatile computer storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0061] The above describes specific embodiments of the present invention. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0062] The above description is only an embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the embodiments of the present invention should be included in the protection scope of the present invention.

Claims

1. A multi-document question answering method based on multi-head self-attention and hierarchical enhancement, characterized in that: The method comprises: Parse the document data in the document library and format it to obtain a document in a preset format; Extracting hierarchical metadata from the preset format document, and constructing a hierarchical document tree based on the hierarchical relationship between the hierarchical metadata, to obtain the contextual relationship in the preset format document; Acquire search information input by a user, and determine one or more target search paths based on the search information; Through the multi-head self-attention embedding module, multi-document retrieval is performed on the documents under each target retrieval path to obtain several relevant document paragraphs; Calculating the comprehensive matching degree between the relevant document paragraphs and the search information, and screening the target document paragraphs according to the comprehensive matching degree; Based on the target document paragraphs and the corresponding contextual relationships, the question-answering results of the retrieval information are sorted out in a large language model.

2. According to claim 1, a multi-document question answering method based on multi-head self-attention and hierarchical enhancement is characterized in that: Parse the document data in the document library and format it to obtain a document in a preset format, including: Importing document data stored in a document library; wherein the document data at least includes plain text data, Word document data, PDF document data, image data, and table data; Based on a large language model and natural language processing technology, the document data is converted into a document in a preset format.

3. According to claim 2, a multi-document question answering method based on multi-head self-attention and hierarchical enhancement is characterized in that: Based on a large language model and natural language processing technology, the document data is converted into a document in a preset format, specifically including: Through the large language model, plain text data, Word document data, and PDF document data are converted into Markdown format documents; For the image data in the PDF document data, use the PDFImageSearcher tool to extract the bitmap and SVG image of the image data, and generate an image description in combination with surrounding text, and generate a descriptive title for the image data based on a visual language model, and generate a Markdown format document based on the descriptive title and the image description; Extracting table titles, row labels, and column labels from the table data through a large language model, generating a semantic representation of the table, and generating a corresponding Markdown format document based on the semantic representation; The generated Markdown format documents are processed for readability through natural language processing technology.

4. The multi-document question answering method based on multi-head self-attention and hierarchical enhancement according to claim 1, characterized in that: Extracting hierarchical metadata from the preset format document, and constructing a hierarchical document tree based on the hierarchical relationship between the hierarchical metadata, and obtaining the contextual relationship in the preset format document, specifically includes: Extracting hierarchical metadata from each preset format document; wherein the hierarchical metadata at least includes various structural elements of the document, and the structural elements at least include chapter titles, chapter subtitles, and paragraphs; Based on the hierarchical relationship between each structural element, a unique identifier is assigned to each structural element; wherein the identifier can be reflected in the hierarchical relationship between other structural elements; According to the identifier, each structural element is constructed into a hierarchical document tree; the root node of the hierarchical document tree is the title of the entire document, and the leaf nodes are each paragraph; The hierarchical document tree is traversed, and based on the connection relationship between each node, the context relationship in the preset format document is added.

5. According to claim 4, a multi-document question answering method based on multi-head self-attention and hierarchical enhancement is characterized in that: Traversing the hierarchical document tree and adding contextual relationships in the preset format document based on the connection relationship between each node, specifically including: Traversing the hierarchical document tree by a depth-first search algorithm, and recording the access order and access path of each node during the traversal process to obtain the contextual relationship between the nodes; Marking the contextual relationship between the nodes in different forms into the data of the preset format document; wherein, adding contextual hierarchical tags to text paragraphs; generating descriptive text for table data and image data, and associating it with related text paragraphs; When the content of the preset format document is updated, the preset format document is re-parsed and the hierarchical document tree is updated.

6. The multi-document question answering method based on multi-head self-attention and hierarchical enhancement according to claim 1, characterized in that: Based on the search information, one or more target search paths are determined, specifically including: Based on different retrieval algorithms, multiple corresponding retrieval paths are constructed to obtain multiple retrieval paths; wherein the different retrieval algorithms at least include: TF-IDF algorithm, BM25 algorithm and vector similarity measurement algorithm; The retrieval information is subjected to path decision by a pre-trained decision router, and one or more target retrieval paths are determined from the multiple retrieval paths for document retrieval.

7. The multi-document question answering method based on multi-head self-attention and hierarchical enhancement according to claim 1, characterized in that: Through the multi-head self-attention embedding module, multi-document retrieval is performed on the documents under each target retrieval path to obtain several relevant document paragraphs, including: Obtaining, by means of the multi-head self-attention embedding module, a retrieval information embedding representation of the retrieval information and a document embedding representation of the document in the preset format; Adding the search information embedding representation to the front of the document embedding representation as an external knowledge anchor point of the document embedding representation to form an embedding combination; The embedding combination is input into a large language model, and a multi-document search is performed according to each target search path to obtain a number of related document paragraphs.

8. The multi-document question answering method based on multi-head self-attention and hierarchical enhancement according to claim 7, characterized in that: Obtaining the retrieval information embedding representation of the retrieval information and the document embedding representation of the preset format document through the multi-head self-attention embedding module specifically includes: The search information and the preset format documents in the document library are represented as text blocks containing a number of tokens, and are mapped to a vector space of fixed dimension through an embedding layer to obtain a corresponding initial embedding representation; The initial embedding representation is divided into a plurality of header data, and attention calculation is performed independently in each header data to obtain a corresponding retrieval information embedding representation and a document embedding representation; Among them, the attention calculation includes at least: linear transformation to generate query, key and value vectors, using dot product attention mechanism to calculate attention weight, weighted summation of value vectors, and concatenating the outputs of all head data and obtaining the final multi-head attention output through linear transformation.

9. The multi-document question answering method based on multi-head self-attention and hierarchical enhancement according to claim 1, characterized in that: Calculating the comprehensive matching degree between the relevant document paragraphs and the search information, and screening the target document paragraphs according to the comprehensive matching degree, specifically including: Converting the retrieved relevant document paragraphs and the retrieval information into vector representations; Based on the vector representation, respectively calculating a relevance score, a previous example similarity score, a neighbor similarity score, and a first example similarity score of each relevant document paragraph and the retrieval information; Performing weighted calculation on the relevance score, the previous case similarity score, the neighbor similarity score, and the first case similarity score to obtain a comprehensive matching degree of each relevant document paragraph; The relevant document paragraphs whose comprehensive matching degree is greater than a preset threshold are determined as the target document paragraphs.

10. A multi-document question-answering device based on multi-head self-attention and hierarchical enhancement, characterized in that: The device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, so that the at least one processor can execute a multi-document question answering method based on multi-head self-attention and hierarchical enhancement according to any one of claims 1-9.

Images