Paragraph logical relationship perception-based super-long document retrieval method, system and product

Abstract

This invention provides a method, system, and product for retrieving ultra-long documents based on paragraph logical relationship awareness, belonging to the field of natural language processing and information retrieval technology. It includes: S1, offline construction of an internal paragraph logical dependency graph of the ultra-long document; S2, when a user initiates a query, retrieving multiple sets of ultra-long documents with pre-constructed internal paragraph logical dependency graphs, returning matching paragraphs and completing their logical dependencies. This invention solves the problem of missing key information caused by neglecting deep logic in traditional retrieval systems by constructing logical dependencies between paragraphs within the document and dynamically completing the context during the retrieval stage.

Description

Technical Field

[0001] This invention belongs to the field of natural language processing and information retrieval technology, specifically relating to a method, system, and product for retrieving ultra-long documents based on paragraph logical relationship awareness. Background Technology

[0002] In the field of information retrieval technology, legal provisions, reports, papers, long articles, and entire books are all considered ultra-long documents (usually much longer than 128K tokens). By using RAG systems designed for ultra-long documents, information retrieval efficiency can be improved, supporting accurate decision-making and research, and promoting knowledge integration and innovation.

[0003] Current retrieval systems for extremely long documents mainly adopt the following paradigms: Block vector retrieval (and preliminary hybrid retrieval): long documents are divided into segments of fixed length and natural length, encoded using a dual-tower model (such as SBERT) and recalled using vector similarity. Some systems are supplemented with keyword retrieval (BM25) and lightweight re-ranking.

[0004] Knowledge graph-enhanced retrieval: Entities, relationships, and events are extracted from documents to construct a structured knowledge graph. During retrieval, it not only relies on vector similarity but also attempts to capture cross-paragraph entity associations and referencing networks through graph traversal to assist in navigating long-form content.

[0005] Static structure assistance: attempts are made to use the document directory for coarse-grained segmentation or to build local knowledge connections, but it often only scratches the surface when dealing with complex logic across chapters.

[0006] However, the above method has the following shortcomings: (1) Logical break: Only the superficially related paragraphs are returned, ignoring the premises, limitations or exceptions on which they depend. For example: when searching for "payment obligation", the results are "the buyer should pay RMB1 million", but "provided that the seller completes the delivery" and "if the delivery is delayed, it will be waived".

[0007] (2) Difficulty in capturing long-range dependencies: Traditional block retrieval breaks long documents into independent segments, destroying the logical connection between chapters; in addition, the dual-tower model lacks bidirectional interaction in the encoding stage, making it difficult to capture cross-chapter references or complex causal chains scattered in different paragraphs, resulting in the misattribution of logical relationships.

[0008] (3) Fully interactive and not scalable: Although the cross encoder can model logic, its computational complexity is O(N). 2 This method is unsuitable for large-scale initial recalls.

[0009] (4) High cost of knowledge graph construction: Directly using large models to construct knowledge graphs increases costs exponentially, which is not conducive to large-scale applications.

[0010] Therefore, there is an urgent need for a long document retrieval scheme that can both efficiently recall data and ensure logical integrity. Summary of the Invention

[0011] The purpose of this invention is to provide a method, system, and product for retrieving ultra-long documents based on paragraph logical relationship awareness. By constructing logical dependencies between paragraphs within the document and dynamically supplementing the context during the retrieval stage, it solves the problem of missing key information caused by traditional retrieval systems ignoring deep logic.

[0012] To achieve the above objectives, the technical solution of the present invention is as follows: A method for retrieving ultra-long documents based on paragraph logical relationship awareness, comprising: S1. Offline construction of internal paragraph logic dependency graphs for ultra-long documents; S2. When a user initiates a query, the system retrieves a collection of very long documents that have been pre-built with internal paragraph logical dependency graphs, returns matching paragraphs, and completes their logical dependency paragraphs.

[0013] Furthermore, in step S1, the offline construction of the internal paragraph logical dependency graph includes: S101. Divide the long document into paragraphs and encode each paragraph independently using a pre-trained encoder; S102. Based on a lightweight paragraph relationship discrimination model, perform fine-grained logical relationship discrimination on the paragraph and output the probability distribution of the relationship category; S103. Based on fine-grained logical relationship discrimination, construct a directed weighted graph for ultra-long documents to explicitly express their internal logical dependency structure.

[0014] Furthermore, in step S101, the pre-trained encoder independently encodes each segment to generate a token-level vector matrix.

[0015] Furthermore, the lightweight paragraph relationship discrimination model described in step S102 includes: Based on the token-level vector matrix of the paragraph pair to be judged, cross-attention calculation is performed; based on the calculated attention weights, token-level feature fusion is performed; a learnable gating mechanism is introduced to dynamically calculate the importance weight of each token and perform weighted summation to obtain the global representation of the paragraph pair; the global representation is input into a feedforward neural network to output the logits of the relation category and calculate the probability distribution of the relation category.

[0016] Preferably, the relationship categories include: premise, causality, progression, contrast, reference, exception, and irrelevant.

[0017] Furthermore, the construction of the directed weighted graph in step S103 includes: Each paragraph corresponds to a node, forming a node set; each node is uniquely identified by its paragraph ID. For each paragraph pair, if the probability of a certain relationship category output by the lightweight paragraph relationship discrimination model is greater than a preset threshold, then a directed edge is added between the two nodes corresponding to the paragraph pair, with the edge weight being the probability of that relationship category. Once the graph is built, it will be serialized and stored.

[0018] Furthermore, step S2 includes: S201. When a user initiates a query, calculate the similarity between the query and all paragraphs in the set of all ultra-long documents with pre-built internal paragraph logical dependency graphs, and recall the most relevant paragraphs globally. S202. Group the recall results by document to form a document-paragraph mapping; S203. Load the internal paragraph logical dependency graph of the document in the document-paragraph mapping, and collect the strong logical dependency paragraphs of each recalled paragraph to form an enhanced subset of the document; S204. Combine all enhanced subsets into a final context set, and synthesize the final context set of multiple ultra-long documents through a large language model to generate the final answer.

[0019] Furthermore, in step S201, the similarity between the query and the paragraph is calculated using a dual-tower model.

[0020] In another aspect, this invention proposes an ultra-long document retrieval system based on paragraph logical relationship awareness, comprising: Graph Construction Module: Offline construction of internal paragraph logic dependency graphs for extremely long documents; Search module: When a user initiates a query, it searches through a collection of very long documents that have been pre-built with internal paragraph logical dependency graphs, returns matching paragraphs and completes their logical dependency paragraphs.

[0021] The present invention also proposes a computer program product, including a computer program that, when executed by a processor, implements the above-described method for retrieving ultra-long documents based on paragraph logical relationship awareness.

[0022] Compared with the prior art, the present invention has the following improvements and beneficial effects: 1. This invention designs a lightweight paragraph relationship discrimination model, which can perform fine-grained logical relationship discrimination on the paragraphs, making the construction of large-scale logical graphs feasible and providing a source of dependent information for retrieval; 2. This invention constructs an internal paragraph logical dependency graph of a document, which can accurately depict the internal dependencies of the document and avoid cross-document noise; 3. This invention implements dynamic completion during retrieval, upgrading the traditional retrieval of "related paragraphs" to "logically complete context," thereby improving the completeness of the answer; 4. The interactive architecture of this invention balances efficiency and accuracy, supports practical deployment, and improves the inference speed of large models. Attached Figure Description

[0023] Figure 1 This is a flowchart of the overall method of Embodiment 1 of the present invention.

[0024] Figure 2 This is a flowchart of the method in Embodiment 2 of the present invention.

[0025] Figure 3 This is a flowchart of the method in Embodiment 3 of the present invention. Detailed Implementation

[0026] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.

[0027] The present invention will now be described in detail with reference to specific embodiments and accompanying drawings.

[0028] Example 1: This embodiment proposes a method for retrieving extremely long documents based on paragraph logical relationship awareness. The overall process is divided into two stages, as follows: Figure 1 As shown, it includes: S1. Offline construction of internal paragraph logic dependency graphs for extremely long documents.

[0029] This step is the first stage, which involves constructing an offline document logic graph.

[0030] To support logic-aware retrieval in subsequent stages, fine-grained logical relationship determination is required for paragraph pairs within the current document. This embodiment constructs an internal paragraph logical dependency graph for each extremely long document during the offline stage.

[0031] To construct the graph, it is necessary to predefine the relationship categories between paragraphs.

[0032] For example, predefined relationship categories include, but are not limited to: premise, causality, progression, contrast, reference, exception, and irrelevance. These relationship categories are defined based on manual annotation rules and legal textual conventions.

[0033] After dividing a very long document into paragraphs, the probability distribution of the relationship categories between paragraphs can be calculated and output based on the paragraph relationship discrimination model.

[0034] The nodes are divided into paragraphs to form a set of nodes. The edges between the nodes are constructed according to the relationship categories between the paragraphs to form a set of edges. The node set and the edge set are used together to construct the internal paragraph logical dependency graph of the document.

[0035] S2. When a user initiates a query, the system retrieves a collection of very long documents that have been pre-built with internal paragraph logical dependency graphs, returns matching paragraphs, and completes their logical dependency paragraphs.

[0036] This step is the second stage, which enables online logical perception retrieval.

[0037] Based on step S1, an internal paragraph logical dependency graph is constructed for the ultra-long document. When a user initiates a query, the system can retrieve multiple sets of long documents with pre-constructed logical graphs.

[0038] The search process includes: Calculate the similarity between the query and all paragraphs in a set of ultra-long documents with pre-built internal paragraph logical dependency graphs, and recall the most relevant paragraphs globally; The recall results are grouped by document to form a document-paragraph mapping; Load the internal paragraph logical dependency graph of the document in the document-paragraph map, and collect its strong logical dependency paragraphs for each recalled paragraph to form an enhanced subset of the document; All enhanced subsets are combined into a final context set. The final context set of multiple very long documents is then integrated through a large language model to generate the final answer.

[0039] The solution in this embodiment can construct a large-scale logical graph of ultra-long documents, accurately depict the internal dependencies of documents, avoid cross-document noise, provide a source of dependency information for retrieval, dynamically complete the data during retrieval, and improve the completeness of the answer; it can achieve both efficient recall and ensure the logical integrity of the answer.

[0040] Example 2: This embodiment is a technical implementation of step S1, offline document logical graph construction, in the method described in Embodiment 1, such as... Figure 2 As shown, it includes: I. Document Slicing and Paragraph Encoding Caching: The input long document is segmented into natural paragraphs (if a natural paragraph is too long, it is further segmented with a fixed length). The paragraphs are represented as follows: , n is the number of paragraphs; a pre-trained encoder (such as DeBERTa-v3) is used for each paragraph. Independent encoding to generate token-level vector matrices This data is then cached in a local index (for example, this can be done locally using tools such as FAISS). This step can be done offline with a time complexity of O(N).

[0041] II. Determining the logical relationships between paragraphs within a document: To support subsequent logic-aware retrieval, fine-grained logical relationship determination is required for paragraph pairs within the current document. This is the core content of this invention.

[0042] To address the issue of excessive computational complexity when dealing with all paragraphs in long documents, this solution introduces a hybrid filtering mechanism combining "local sliding window + global key nodes": that is, only each segment is evaluated against the set. The relationship between elements in paragraphs, paragraphs The candidate discrimination segment set is ∪ ∪ The complexity is O(N).

[0043] This embodiment designs a lightweight Paragraph Relation Discriminator (PRD) model. This model takes the token-level vector matrix of the target paragraph and any of its candidate paragraphs as input and outputs a probability distribution of predefined relation categories. The model has approximately 830K parameters, excluding the underlying encoder. The encoder base can be a model such as DeBERTa-v3.

[0044] The lightweight paragraph relationship discrimination model PRD will be explained in detail below.

[0045] Let the paragraphs to be judged be... and The token-level vector matrices obtained after encoding by the pre-trained encoder are as follows: and : ,in For paragraph The number of tokens; d For example, the encoder's hidden layer dimension. d The possible value is 768; ,in For paragraph The number of tokens.

[0046] During the training and inference process of PRD, and It is treated as a fixed input (i.e., the underlying encoder parameters are frozen).

[0047] The lightweight paragraph relationship discrimination model PRD consists of the following four consecutive steps: 1. Cross-attention calculation: Calculate paragraphs right Attention weights, and Information alignment injection Context: ; ; in express transpose; This is the scaling factor; The middle element is ,express The i-th token pair The attention level of the j-th token; The middle element is ,express Fusion at the i-th position The representation following the information.

[0048] There are no additional trainable parameters for this step.

[0049] 2. Token-level feature fusion: right Each token position i (i=1,..., (), splicing three signals to explicitly model semantic interactions: ; in: ; for paragraph No. The original vector representation of each token; ; represents the context vector of the same position after cross-attention weighting; This represents element-wise multiplication and is used to capture the interaction signals between two sets of representations (such as the simultaneous activation of "cause" and "therefore"), which is crucial for identifying logical relationships such as causality and premises. The concatenated vector has a dimension of 3d = 2304. This serves as a fusion feature representation of the token.

[0050] 3. Gated weighted pooling: A learnable gating mechanism is introduced to dynamically calculate the importance weight of each token, and the weighted sum is used to obtain the paragraph's global representation: ; ； ; in: The weight matrix is ​​a linear transformation matrix; It is the bias vector; It is a context vector. =128 is the gated hidden layer dimension (can be configured according to requirements); , , All of these are trainable parameters; p represents the global representation.

[0051] This mechanism enables the model to automatically focus on discriminative tokens (such as logical connectors) and suppress filler words or irrelevant words.

[0052] 4. Relationship Classification Header: Input the global representation p into a two-layer feedforward neural network, and it outputs logits of K types of relationships: ; ; in: ; ; ; ; = 512 is the dimension of the intermediate hidden layer; K is the number of predefined relation categories (e.g., K = 7); All parameters , , , All are trainable; the final relation probability is obtained through... get.

[0053] Predefined relation categories include, but are not limited to: premise, causality, progression, contrast, reference, exception, and irrelevance.

[0054] Among them, "prerequisite" refers to The necessary conditions for the establishment of the agreement are as follows: (e.g., "the seller's completion of delivery" is a prerequisite for "the buyer's payment"). "Exception" refers to right The scope of application is limited, for example: "force majeure" exempts payment obligations; "Cause and effect" refers to lead to occur; "Quote" refers to Clearly pointing For example, "See Article 5"; "Progressive, contrastive, irrelevant": used to cover other common semantic relationships.

[0055] All relationship categories are defined based on manual annotation rules and legal textual conventions.

[0056] III. Constructing and storing the logical dependency graph of paragraphs within the document: After determining the relationships between paragraph pairs within each document, a directed weighted graph G=(V,E) is constructed for each document to explicitly express its internal logical dependency structure. This graph is generated solely based on the current document content and is completely isolated from other extremely long documents.

[0057] The construction of the map specifically includes the following elements: 1. Node set V: Each original paragraph Corresponding to a node , , where n is the number of paragraphs in the current document. Nodes are uniquely identified by paragraph IDs (for example, paragraph numbers or hash values).

[0058] 2. Edge set E: The rules for constructing edge set E include: for each pair of paragraphs ( , )( m ≠ n If the probability of a certain relation category r (including premise, causation, reference, exception) output by the PRD model is greater than the preset threshold τ (default τ=0.7), then add a record from... point to The directed edges; The direction of the edge is determined by semantics, for example: yes "The premise is", then the edge is → ; 3. Edge weight : The probability output by the PRD model corresponding to the relationship category r; For example, in order to control the graph density, this embodiment retains at most the top-3 in-degree and top-3 out-degree edges for each node.

[0059] 4. Graph Storage: After construction, the graph is serialized into JSON or Protocol Buffer format and associated with the original document via document ID, then stored on the local file system or cloud object storage (such as Amazon S3). The stored content includes: Node list: Contains only paragraph IDs as unique identifiers, such as 1, 2, ..., n; Edge list: Each edge records the source node ID, target node ID, relationship type (string), and weight (floating-point number); Meta-information: Document ID, graph construction timestamp, and version number of the PRD model used.

[0060] Using the above elements, the internal paragraph logical dependency graph of the document was constructed.

[0061] In this embodiment, the PRD model enables the construction of large-scale logical graphs, providing a source of dependency information for retrieval; by constructing internal paragraph logical dependency graphs, the internal dependencies of documents are accurately characterized, avoiding cross-document noise and providing support for the next stage of online logic-aware retrieval.

[0062] Example 3: This embodiment is a technical implementation of step S2, online logical perception retrieval, in the method described in Embodiment 1.

[0063] When a user initiates a query (Q), multiple long documents with pre-built internal paragraph logical dependency graphs can be retrieved. The processing flow is as follows: Figure 3 As shown, it includes: (a) Initial cross-document recall: The similarity between query Q and all paragraphs in a long document set with all pre-built internal paragraph logical dependency graphs is calculated using a dual-tower model, and the most relevant paragraphs in the global Top-K (e.g., K=10) are retrieved.

[0064] Simultaneously, the document ID of the document to which each recalled segment belongs (i.e., the hit document) is recorded, resulting in several (documents) Paragraph As a result of the recall, among which Indicates the recall paragraph. This indicates the document to which the recalled paragraph belongs.

[0065] (ii) Grouping by document: The recall results are grouped by document ID to form a document-paragraph mapping: .

[0066] (III) Document-by-document logical completion: This is the core step of online logical perception retrieval.

[0067] For each hit document , load The corresponding dedicated internal paragraph logic dependency graph, and then... Each recall paragraph Locate the corresponding node in the graph; extend that node by 1 hop along the graph, or 2 hops if necessary. For example, if the neighboring node of the 1-hop contains a "reference" relationship category, it is necessary to continue to extend the dependency of the referenced paragraph. Collect the strongly logically dependent paragraphs of the node obtained after expanding along the graph by 1 hop or 2 hops. This includes incoming neighbor nodes, such as supporting paragraphs like "premise," "exception," and "cause and effect." It also includes outgoing neighbor nodes, which are only included under "reference" relationships, i.e., paragraphs. Specific terms; paragraph The document and its dependent paragraphs are merged into this document. Enhanced subset ; right Remove duplicates and sort by original text order or edge weight.

[0068] (iv) Cross-document result fusion: Combine the enhanced subsets of all hit documents into the final context set E: .

[0069] Output format example: [Document D1] Paragraph 3: The seller shall complete the asset divestiture on the closing date.

[0070] Paragraph 8: The buyer shall pay the initial payment within 5 business days after the seller completes the asset divestiture. (Prerequisite relationship) Paragraph 22: If the delay is caused by force majeure, the payment obligation shall be postponed accordingly. (Exception) [Document D5] Paragraph 9: This Agreement shall be governed by the laws of New York State.

[0071] Paragraph 14: The dispute should be submitted to the U.S. District Court for the Southern District of New York. (Reference) The final context set E, which is enhanced by the above structure, is used as a prompt and input into the Large Language Model (LLM). The LLM then integrates information from multiple documents to generate the final answer.

[0072] The solution described in this embodiment achieves dynamic completion when retrieving extremely long documents, upgrades the search results to a logically complete context, and adopts a caching + sparse interaction architecture to balance efficiency and accuracy, supporting actual deployment.

[0073] Example 4: This embodiment is an application example of the methods described in Embodiments 1-3. The methods described in Embodiments 1-3 are particularly suitable for very long documents with explicit logical dependencies between paragraphs, such as legal contracts like merger and acquisition agreements and lease contracts, or international standards like ISO series, or legal clauses and regulatory rules. For documents with strong paragraph independence, such as news articles and instruction manuals, they can be used selectively. This embodiment uses legal contract question-and-answer retrieval as the application scenario, and the implementation process is as follows: 1. User-uploaded documents: For example, a merger and acquisition agreement. The system automatically triggers the offline map construction process.

[0074] 2. Offline construction of logical graph: The document is segmented into paragraphs, encoded using DeBERTa-v3, and the vectors are cached.

[0075] The PRD model is used to determine the logical relationships (premise, exception, reference, etc.) between paragraph pairs, and a directed weighted graph is constructed.

[0076] The graph is stored in JSON format and associated with the original document via document ID.

[0077] 3. Online search and completion: When a user queries "What representations and warranties does the seller make?", the system retrieves directly matching paragraphs (such as "The seller declares that the assets are free from defects").

[0078] Automatically complete logically dependent paragraphs (such as "indemnification for breach of the statement" and "disclaimer") to create enhanced context.

[0079] The enhanced context input LLM generates structured answers, for example: Statement of Terms: The seller declares that the assets are free from defects (paragraph 3); Consequences of breach: The buyer has the right to terminate the contract and claim compensation (paragraph 12); Disclaimer: If the defect in the asset is caused by the buyer's fault (paragraph 18); 4. Dynamic control mechanism: The system supports dynamically deciding whether to enable logical completion based on the query intent (such as the presence of logical connectors "if" or "unless") or the document type.

[0080] 5. Construction of PRD model training data: 1) Data Units and Sources: Data unit: paragraph pairs within the document (180,000 pairs in total), each pair containing two paragraphs and their logical relationship tags.

[0081] Dataset: LegalDocRel (2,000 real business contracts covering scenarios such as mergers and acquisitions, leasing, and employment).

[0082] 2) Labeling process: LLM initial labeling: Use large language models such as GPT-4 to generate preliminary relation labels (such as "cause and effect" and "exceptions") for paragraph pairs.

[0083] Rule filtering: Only keep paragraph pairs containing logical connectors (such as "if", "unless", "but", "therefore"); exclude paragraphs that are too short (<10 words) or are purely formatted.

[0084] Manual verification: Label consistency is verified by legal professionals to ensure that the Kappa value is > 0.82 (indicating a high degree of consistency among labelers).

[0085] The final dataset, LegalDocRel, contains 180,000 annotated paragraph pairs, covering 7 types of relations (presupposition, causality, exception, reference, progression, contrast, and irrelevance).

[0086] 3) Loss function: To address the problem of severely imbalanced category distribution (the "irrelevant" category accounts for more than 60% of the samples, while key supporting relationship categories such as "premise" and "exception" account for less than 10%), this embodiment uses Focal Loss with category weights as a unified loss function.

[0087] Standard cross-entropy loss treats all classes equally, which can easily lead to model bias towards the majority class in highly imbalanced data. Therefore, two core mechanisms of Focal Loss are introduced: Modulation factor Automatically reduces the loss contribution of easily separable samples (such as a large number of "irrelevant" samples).

[0088] Category weight Explicitly assign higher learning priority to a few key categories.

[0089] The final loss function is defined as follows: ; in: K Total number of categories; For one-hot tags; The probability predicted by the model; Used to control the degree of focus on samples of varying difficulty; For category Learnable or preset weights.

[0090] 4) Category balancing strategy: Inverse frequency weighting is set based on the frequency of each category in the training set and further adjusted: For the "irrelevant" category (accounting for 60%+), let... ; For low-frequency but high-value categories such as "preconditions" and "exceptions" (each <8%), let... ; Other categories are set .

[0091] This design ensures that the model focuses on both hard sample identification and class importance balance during the optimization process. Experiments show that it significantly improves the recall rate of key relationship types (especially the "premise" and "exception" classes), while avoiding gradient conflicts or weight imbalances caused by simply mixing the two loss functions.

[0092] 5) Validation set partitioning: 10% of LegalDocRel is allocated as the validation set, divided by document granularity (to avoid training / validation overlap of the same document). Validation metrics: F1 score (weighted average), AUC-ROC (multi-label); Early stopping condition: F1 score does not improve after 3 consecutive epochs of verification.

[0093] 6) Training parameters: Optimizer: AdamW (learning rate) Weight decay ); Batch size: 32; Hardware: 8 × A100 GPUs.

[0094] 7) Verification and Results: On the LegalQA-v1 test set, the PRD model achieved an F1 score of 89.2% for the "Premise" and "Exception" classes, significantly outperforming the SBERT+BM25 baseline (62.1%).

[0095] Training time: On an 8×A100 GPU, training the LegalDocRel dataset takes approximately 2 hours (batch size 32, learning rate 2e-4).

[0096] 8) System Deployment: Supports document-level parallel processing; After a new document is uploaded, offline map construction is automatically triggered; The average query response time is 1.6 seconds (including the completion process, 100 tests).

[0097] Example 5: This embodiment proposes an ultra-long document retrieval system based on paragraph logical relationship awareness, including: Graph Construction Module: Offline construction of internal paragraph logic dependency graphs for extremely long documents; Search module: When a user initiates a query, it searches through a collection of very long documents that have been pre-built with internal paragraph logical dependency graphs, returns matching paragraphs and completes their logical dependency paragraphs.

[0098] The offline construction of the graph construction module, which includes the internal paragraph logic dependency graph, includes: The very long document is divided into paragraphs and each paragraph is encoded independently using a pre-trained encoder; A lightweight paragraph relationship discrimination model is used to perform fine-grained logical relationship discrimination on the paragraphs and output the probability distribution of relationship categories; Based on fine-grained logical relationship discrimination, a directed weighted graph is constructed for ultra-long documents to explicitly express their internal logical dependency structure.

[0099] Specifically, the pre-trained encoder independently encodes each segment to generate a token-level vector matrix.

[0100] The lightweight paragraph relationship discrimination model includes: Based on the token-level vector matrix of the paragraph pair to be judged, cross-attention calculation is performed; based on the calculated attention weights, token-level feature fusion is performed; a learnable gating mechanism is introduced to dynamically calculate the importance weight of each token and perform weighted summation to obtain the global representation of the paragraph pair; the global representation is input into a feedforward neural network to output the logits of the relation category and calculate the probability distribution of the relation category.

[0101] The relationship categories include: premise, causality, progression, contrast, reference, exception, and irrelevance.

[0102] The construction of the directed weighted graph includes: Each paragraph corresponds to a node, forming a node set; each node is uniquely identified by its paragraph ID. For each paragraph pair, if the probability of a certain relationship category output by the lightweight paragraph relationship discrimination model is greater than a preset threshold, then a directed edge is added between the two nodes corresponding to the paragraph pair, with the edge weight being the probability of that relationship category. Once the graph is built, it will be serialized and stored.

[0103] The search module includes: When a user initiates a query, the similarity between the query and all paragraphs in the set of ultra-long documents with pre-built internal paragraph logical dependency graphs is calculated, and the most relevant paragraphs in the global dataset are retrieved. The recall results are grouped by document to form a document-paragraph mapping; Load the internal paragraph logical dependency graph of the document in the document-paragraph map, and collect its strong logical dependency paragraphs for each recalled paragraph to form an enhanced subset of the document; All enhanced subsets are combined into a final context set. The final context set of multiple very long documents is then integrated through a large language model to generate the final answer.

[0104] The dual-tower model can be used to calculate the similarity between a query and a paragraph.

[0105] The ultra-long document retrieval system based on paragraph logical relationship awareness proposed in this embodiment can achieve the ultra-long document retrieval method based on paragraph logical relationship awareness described in Embodiments 1-3, and has the same beneficial effects.

[0106] The above-described embodiments are merely preferred embodiments of the present invention and are only used to help understand the method and core ideas of this application. The scope of protection of the present invention is not limited to the above embodiments, and all technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A method for retrieving extremely long documents based on paragraph logical relationship awareness, characterized in that, include: S1. Offline construction of internal paragraph logic dependency graphs for ultra-long documents; S2. When a user initiates a query, the system retrieves a collection of very long documents that have been pre-built with internal paragraph logical dependency graphs, returns matching paragraphs, and completes their logical dependency paragraphs.

2. The method for retrieving ultra-long documents based on paragraph logical relationship awareness according to claim 1, characterized in that, In step S1, the offline construction of the internal paragraph logical dependency graph includes: S101. Divide the long document into paragraphs and encode each paragraph independently using a pre-trained encoder; S102. Based on a lightweight paragraph relationship discrimination model, perform fine-grained logical relationship discrimination on the paragraph and output the probability distribution of the relationship category; S103. Based on fine-grained logical relationship discrimination, construct a directed weighted graph for ultra-long documents to explicitly express their internal logical dependency structure.

3. The method for retrieving ultra-long documents based on paragraph logical relationship awareness according to claim 2, characterized in that, In step S101, the pre-trained encoder independently encodes each segment to generate a token-level vector matrix.

4. The method for retrieving ultra-long documents based on paragraph logical relationship awareness according to claim 3, characterized in that, The lightweight paragraph relationship discrimination model described in step S102 includes: Based on the token-level vector matrix of the paragraph pair to be judged, cross-attention calculation is performed; based on the calculated attention weights, token-level feature fusion is performed; a learnable gating mechanism is introduced to dynamically calculate the importance weight of each token and perform weighted summation to obtain the global representation of the paragraph pair; the global representation is input into a feedforward neural network to output the logits of the relation category and calculate the probability distribution of the relation category.

5. The method for retrieving ultra-long documents based on paragraph logical relationship awareness according to claim 2, characterized in that, The relationship categories include: premise, causality, progression, contrast, reference, exception, and irrelevance.

6. The method for retrieving ultra-long documents based on paragraph logical relationship awareness according to claim 2, characterized in that, The construction of the directed weighted graph in step S103 includes: Each paragraph corresponds to a node, forming a node set; each node is uniquely identified by its paragraph ID. For each paragraph pair, if the probability of a certain relationship category output by the lightweight paragraph relationship discrimination model is greater than a preset threshold, then a directed edge is added between the two nodes corresponding to the paragraph pair, with the edge weight being the probability of that relationship category. Once the graph is built, it will be serialized and stored.

7. The method for retrieving ultra-long documents based on paragraph logical relationship awareness according to claim 1, characterized in that, Step S2 includes: S201. When a user initiates a query, calculate the similarity between the query and all paragraphs in the set of all ultra-long documents with pre-built internal paragraph logical dependency graphs, and recall the most relevant paragraphs globally. S202. Group the recall results by document to form a document-paragraph mapping; S203. Load the internal paragraph logical dependency graph of the document in the document-paragraph mapping, and collect the strong logical dependency paragraphs of each recalled paragraph to form an enhanced subset of the document; S204. Combine all enhanced subsets into a final context set, and synthesize the final context set of multiple ultra-long documents through a large language model to generate the final answer.

8. The method for retrieving ultra-long documents based on paragraph logical relationship awareness according to claim 7, characterized in that, In step S201, the similarity between the query and the paragraph is calculated using the dual-tower model.

9. A long document retrieval system based on paragraph logical relationship awareness, characterized in that, include: Graph Construction Module: Offline construction of internal paragraph logic dependency graphs for extremely long documents; Search module: When a user initiates a query, it searches through a collection of very long documents that have been pre-built with internal paragraph logical dependency graphs, returns matching paragraphs and completes their logical dependency paragraphs.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the ultra-long document retrieval method based on paragraph logical relationship awareness as described in any one of claims 1-8.

Images