A method and device for improving the recall effect of RAG

Abstract

The application provides a method and device for improving RAG recall effect, and belongs to the technical field of computers, which comprises: document input and layout structure analysis; recognizing a layout unit of an input document and extracting text content in the layout unit to generate associated data of layout and content; double-dimension relationship graph construction; performing semantic segmentation based on the layout unit and extracting logical relationship of the layout unit; constructing a double-dimension relationship graph of semantic relationship and layout relationship; multi-dimension information fusion recall; receiving a user query and performing semantic analysis, recalling similar semantic slices from a semantic community and a layout community, performing double-graph cross verification, dynamically adjusting weights, and calculating and obtaining a final recall result. The application optimizes the knowledge base construction mode of RAG from the perspective of layout, fuses the multi-dimension relationship of text semantics and layout logic, makes the information association in the knowledge base more comprehensive, and can simultaneously depend on semantic similarity and layout logic during retrieval and recall, so that the recall effect is significantly improved.

Description

Technical Field

[0001] This invention belongs to the field of computer technology, and specifically relates to a method and apparatus for improving the recall effect of RAG (Recall Aid Group). Background Technology

[0002] The core workflow of current Retrieval-Augmented Generation (RAG) technology mainly includes three stages: document preprocessing, knowledge base construction, and retrieval recall. In the document preprocessing stage, existing technologies generally use methods such as fixed-length segmentation, semantic window segmentation, or paragraph-level segmentation to break down the input unstructured document (such as PDF, PPT, TXT, etc.) into several text chunks. In the knowledge base construction stage, the text chunks are typically converted into vector form and stored in a vector database. In the retrieval recall stage, the similarity between the user query and the text chunk vectors is calculated to recall chunks with high relevance as supplementary context for the Large Language Model (LLM), thereby improving the accuracy and reliability of the model's output.

[0003] To further improve the accuracy of retrieval results, the industry has proposed RAG optimization schemes based on knowledge graphs, with Microsoft's GraphRAG (Graph-Based Retrieval-Augmented Generation) being a representative example. The core idea of ​​GraphRAG is to extract entities, concepts, and their semantic relationships from text content using natural language processing techniques (such as entity recognition and relation extraction) on top of traditional text slicing, and then construct an entity relationship graph based on these relationships. During retrieval, GraphRAG not only relies on vector similarity calculations but also utilizes the connection path information of entities in the relationship graph to achieve more logical retrieval, such as locating more relevant text slices through the subordinate or causal relationships between entities. However, GraphRAG's relation construction relies entirely on the semantic analysis of the text content itself, mining information relationships only from a textual perspective, without considering the inherent layout features of the document.

[0004] Existing RAG solutions suffer from significant technical problems. They fail to capture and utilize the layout information of input documents, resulting in a single dimension of information association in the knowledge base and limited recall effectiveness. Specifically, when users understand a document normally, the layout is a crucial basis for obtaining content associations—for example, the hierarchical relationship (subordination) between headings and body text, the parallel relationship between headings at the same level, the correspondence between cell content in a table, and the association between images and captions. This layout information can intuitively reflect the logical structure between text blocks. However, existing technologies only focus on the semantic association of text content and cannot identify the aforementioned layout-level relationships. Even optimization solutions like GraphRAG, which are based on knowledge graphs, limit the construction of their relationship graphs to textual semantics and fail to integrate layout-level information. With the maturity of object recognition technology and multimodal technology, the ability to reliably parse document layout structure has been achieved. However, existing RAG technology has not yet applied this capability to knowledge base construction, resulting in a knowledge base that only contains textual semantic associations, lacking logical layout associations, and having insufficient information completeness, ultimately affecting the accuracy and comprehensiveness of slice recall. Summary of the Invention

[0005] The purpose of this invention is to provide a method and apparatus for improving the recall effect of RAG, which can optimize the knowledge base construction method of RAG from the perspective of layout, integrate the multi-dimensional relationship between text semantics and layout logic, make the information in the knowledge base more comprehensive, and can simultaneously rely on semantic similarity and layout logic during retrieval and recall, thereby significantly improving the recall effect.

[0006] To achieve the above objectives, the technical solution of the present invention is as follows:

[0007] A method to improve RAG recall effectiveness includes:

[0008] S1. Document Input and Layout Structure Analysis: Identify layout units in the input document and extract the text content within the layout units to generate correlation data between layout and content;

[0009] S2. Two-dimensional relationship graph construction: Semantic segmentation based on typesetting units, and extraction of logical relationships between typesetting units based on the associated data; construction of a two-dimensional relationship graph of semantic and typesetting relationships; division of semantic communities and typesetting communities and storage of information;

[0010] S3, Multi-dimensional Information Fusion Recall: Receive user queries and perform semantic parsing, recall similar semantic slices from semantic and typesetting communities, perform dual-graph cross-validation, dynamically adjust weights, calculate and obtain the final recall results.

[0011] Furthermore, step S1 includes:

[0012] S101. Multi-format document input: Receives unstructured documents uploaded by users, supporting multiple document formats including PDF, PPT, and Word.

[0013] S102, BBOX Recognition: The document layout analysis model based on object detection is used to perform page-level parsing of the input document, identify and output each layout unit;

[0014] S103, OCR Text Extraction: For each identified layout unit, a high-precision OCR model is called to extract the text content within the unit, generating associated data including BBOX ID, layout type, coordinate information, and text content, providing basic data for subsequent relationship analysis and semantic processing.

[0015] Furthermore, step S102 specifically includes:

[0016] S102-1. Locate all visually independent content areas within the page and output the pixel coordinates of each area;

[0017] S102-2. Classify each area by type. Use classification tags, including first-level headings, second-level headings, body paragraphs, tables, pictures, figure captions, and formulas, to achieve structured decomposition of document content and obtain layout units.

[0018] Furthermore, step S2 includes:

[0019] S201. Modal model identification of layout logic relationship: Input the associated data into the document multimodal model, and analyze the logical relationship between each layout unit by integrating the three modal information of text content, BBOX coordinates and layout type, and construct a layout relationship diagram;

[0020] S202, Semantic Segmentation and Text Processing: Semantic segmentation is performed based on the typesetting unit. The independent typesetting unit is used as the basic semantic unit. If the text length of the typesetting unit exceeds the set threshold, a second segmentation is performed according to semantic integrity to generate the final text slice. Each text slice contains complete semantics and is associated with the original typesetting unit information BBOX ID.

[0021] S203. Construct a semantic relationship graph: For the text slices, construct a semantic relationship graph using the core process of the graph-enhanced retrieval generation system GraphRAG, and combine it with the layout relationship graph to achieve the construction of a two-dimensional relationship graph;

[0022] S204, Community Detection: Using a community clustering algorithm based on modular exponent Q, communities are divided into layout relationship graph and semantic relationship graph respectively. A summary is generated for layout community and semantic community respectively, and a summary of the meta-information of all nodes and edges in the community is recorded.

[0023] S205. Information storage: Use a vector database to record the vector information of the original text of the text slice, and use a graph database to record the relevant information of the two-dimensional relationship graph to form a knowledge base for subsequent retrieval.

[0024] Furthermore, step S201 specifically includes:

[0025] S201-1, Perform feature learning on the spatial location and type hierarchy of BBOX within the page;

[0026] S201-2. Output layout relation triples in the following format: source BBOX ID - relation type - target BBOX ID.

[0027] Furthermore, step S203 specifically includes:

[0028] S203-1. Perform entity recognition and relation extraction on each text slice to extract entities / concepts and semantic associations;

[0029] S203-2. Construct a semantic relationship graph with entities / concepts as nodes and semantic relationships as edges. Node attributes include entity type and associated Chunk ID, and edge attributes are semantic relationship types.

[0030] S203-3. Establish a connection between the semantic relationship diagram and the layout relationship diagram through "Chunk ID-BBOX ID" to realize the construction of a two-dimensional relationship diagram.

[0031] Furthermore, step S3 includes:

[0032] S301. User Query Preprocessing: Receive user query, perform semantic parsing on it, including entity extraction and intent recognition, and convert the query into vector form.

[0033] S302, Initial Screening: Vector Similarity Recall. Calculate the vector similarity between the Query vector and all text slice vectors in the knowledge base, and recall the Top-N candidate text slices with the highest similarity. Simultaneously, retrieve semantic communities based on the Query entity, calculate the semantic community association degree between the Query and the semantic community, and recall the X semantic communities with the highest association degree and their contained text slices. Combine the implicit layout requirements of the Query to retrieve layout communities, calculate the layout community association degree between the Query and the layout community, and recall Y layout communities and their contained text slices. Merge these to form a candidate Chunk pool as the basic recall result.

[0034] S303, Fine-tuning: Includes the following stages:

[0035] Cross-validation phase: Perform bi-graph cross-validation on the candidate Chunk pool to obtain the cross-validation bonus value;

[0036] Dynamic weight ranking stage: Based on the graph complexity index, the weight ratio of layout and semantic relationship is dynamically adjusted to obtain semantic weight and layout weight;

[0037] Comprehensive score calculation stage: Calculate the comprehensive score of the text slices. Comprehensive score = (vector similarity × 0.3) + (semantic community relevance × semantic weight) + (typography community relevance × typography weight) + (cross-validation bonus). The top few scores in descending order are taken as the final recall results.

[0038] S304. Output Results: The final recall results of the text slices are used as supplementary context and input into the large language model along with the user query to generate a response.

[0039] Preferably, in step S303, the dual-graph cross-validation includes:

[0040] If a text slice exists in both the semantic community recall results and the typesetting community recall results, it is marked as a dual-graph cross-validation chunk, and a cross-validation bonus value is set. Then, the mapping consistency between the semantic community and the typesetting community to which the text slice belongs is checked. If the topic directions of the two match, the cross-validation bonus value is further increased.

[0041] Preferably, in step S303, the dynamic adjustment of the weight ratio of layout and semantic relationship based on graph complexity index includes: if the complexity of the layout relationship graph is greater than the complexity of the semantic relationship graph, the layout weight is higher than the semantic weight; if the complexity of the layout relationship graph is less than the complexity of the semantic relationship graph, the semantic weight is higher than the layout weight; otherwise, the two weights are equal.

[0042] In another aspect, the present invention also proposes an apparatus for improving the recall effect of RAG, comprising:

[0043] Document Input and Layout Structure Parsing Module: Identifies layout units in the input document and extracts the text content within the layout units, generating correlation data between layout and content;

[0044] Two-dimensional relationship graph construction module: performs semantic segmentation based on layout units, and extracts the logical relationships of layout units based on the associated data; constructs a two-dimensional relationship graph of semantic relationships and layout relationships; divides semantic communities and layout communities and stores information;

[0045] Multi-dimensional information fusion recall module: Receives user queries and performs semantic parsing, recalls similar semantic slices from semantic and typesetting communities, performs dual-graph cross-validation, dynamically adjusts weights, calculates and obtains the final recall results.

[0046] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0047] This invention optimizes the knowledge base construction method of RAG from a layout perspective: it utilizes object recognition technology to analyze the layout structure of input documents (such as PDFs and PPTs), dividing the document into different layout units such as title blocks, body text blocks, table blocks, and image blocks; then, it analyzes the logical relationships (such as subordinate, parallel, and corresponding relationships) between each layout unit through a multimodal model; when constructing the knowledge base, it not only stores the vector information and semantic associations of text slices, but also records the structural relationships of each layout unit, forming a multi-dimensional relationship graph that integrates text semantics and layout logic. In this way, the information associations in the knowledge base are more comprehensive, and retrieval and recall can be based on both semantic similarity and layout logic simultaneously, thereby significantly improving the recall effect. Attached Figure Description

[0048] Figure 1 This is a flowchart illustrating Embodiment 1 of the present invention;

[0049] Figure 2 This is a schematic diagram of the data processing and relationship graph construction process of Embodiment 2 of the present invention;

[0050] Figure 3 This is a schematic diagram of the recall text flow in Embodiment 3 of the present invention;

[0051] Figure 4 This is a structural diagram of Embodiment 4 of the present invention. Detailed Implementation

[0052] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.

[0053] The present invention will now be described in detail with reference to specific embodiments and accompanying drawings.

[0054] Example 1:

[0055] This embodiment proposes a method to improve the recall effect of RAG, such as Figure 1 As shown, it includes:

[0056] A method to improve RAG recall effectiveness includes:

[0057] S1. Document Input and Layout Structure Analysis: Identify layout units in the input document and extract the text content within the layout units to generate correlation data between layout and content.

[0058] This step primarily employs document block division (BBOX) technology and optical character recognition (OCR) technology.

[0059] Document Block Occlusion (BBOX) technology is used to divide a document page into independent semantic regions (i.e., bounding boxes) according to its layout structure, and is a key step in capturing layout information. Its core is based on object detection algorithms (such as YOLOv8 and Faster R-CNN) or document-specific layout analysis models. Through pixel-level feature learning of the document page, it automatically identifies and divides layout units with different functions, such as titles, body text, tables, images, formulas, and captions. The segmentation results are output in the form of "bounding box coordinates + region type," where the bounding box coordinates (such as the x / y coordinates of the top-left corner and the x / y coordinates of the bottom-right corner) accurately reflect the spatial position of each layout unit on the page, providing spatial dimensional basis for subsequent analysis of the logical relationships between units. Currently, this technology is widely used in office software (such as Adobe Acrobat and WPS), supporting layout segmentation for mainstream document formats such as PDF, PPT, and Word, with an accuracy rate of over 95%.

[0060] Optical Character Recognition (OCR) technology is fundamental to document text information extraction, capable of converting printed or handwritten text in scanned documents and image-based documents (such as PDF scans and PPT screenshots) into editable digital text. Currently, mainstream OCR technologies boast high accuracy (over 99% character recognition accuracy in general scenarios), multilingual support, and adaptability to complex backgrounds, handling complex document scenarios such as tilted, blurred, and low-resolution documents. For example, deep learning-based OCR models (such as PP-OCR and Tesseract 5.0) employ a two-stage task of text detection and text recognition. First, they locate the text region, then accurately extract the text content, providing raw text data for subsequent semantic analysis in layout units.

[0061] S2. Construction of a two-dimensional relationship graph: Semantic segmentation is performed based on typesetting units, and logical relationships between typesetting units are extracted based on the associated data; a two-dimensional relationship graph of semantic relationships and typesetting relationships is constructed; semantic communities and typesetting communities are divided and information is stored.

[0062] The core of this step is achieved through multimodal model layout logic relationship recognition technology.

[0063] In multimodal model-based layout logic relationship recognition technology, document multimodal models possess the ability to integrate textual and spatial layout information, making them a core technology for analyzing the logical relationships between layout units. Currently mature document multimodal models (such as the LayoutLM series and DocLlama) learn document layout rules by introducing multimodal inputs of "text content + position coordinates + region type" during the pre-training stage. For example, they identify subordinate relationships by recognizing "heading areas immediately below the body text area," parallel relationships by recognizing "heading areas of the same level with equal width and vertical distribution," and correspondence relationships by recognizing "image areas and caption areas being adjacent in position and semantically related." These models, through pre-training tasks such as layout prediction and text-region matching, possess powerful layout logic understanding capabilities, capable of outputting the relationship types between each BBOX (such as subordinate, parallel, inclusive, and corresponding), with relationship recognition accuracy exceeding 90% in general document scenarios. This provides direct technical support for constructing a multi-dimensional relationship graph of "text semantics + layout logic."

[0064] S3, Multi-dimensional Information Fusion Recall: Receive user queries and perform semantic parsing, recall similar semantic slices from semantic and typesetting communities, perform dual-graph cross-validation, dynamically adjust weights, calculate and obtain the final recall results.

[0065] Based on the above steps, not only are the vector information and semantic relationships of text slices stored, but the structural relationships of each typesetting unit are also recorded, forming a multi-dimensional relationship graph that integrates text semantics and typesetting logic. In this way, the information relationships in the knowledge base are more comprehensive, and retrieval and recall can be based on both semantic similarity and typesetting logic simultaneously, thereby significantly improving the recall effect.

[0066] Example 2:

[0067] This embodiment is a detailed application description of the data processing and relationship graph construction process in steps S1 and S2 of Embodiment 1.

[0068] like Figure 2 As shown, the specific content is as follows:

[0069] Phase 1: Document Input and Layout Structure Analysis.

[0070] Step 1.1, Inputting multi-format documents:

[0071] It receives unstructured documents uploaded by users, supporting mainstream formats including PDF (including scanned copies), PPT, Word, etc., as the data processing source in this embodiment.

[0072] Step 1.2, BBOX recognition of layout units:

[0073] The document layout analysis model based on object detection (such as YOLOv8-Doc and LayoutParser) is used to perform page-level parsing of the input document, identify and output the bounding boxes (BBOX) and type labels of each layout unit.

[0074] Specifically, it includes:

[0075] (1) Locate all visually independent content areas within the page and output the pixel coordinates of each area (top left x1, y1 - bottom right x2, y2).

[0076] (2) Classify each area by type. For example, the classification labels may include “first-level heading”, “second-level heading”, “body paragraph”, “table”, “picture”, “caption”, “formula”, etc., to realize the structured splitting of document content.

[0077] Step 1.3, OCR Text Extraction: For each identified BBOX layout unit, excluding pure image units, a high-precision OCR model (such as PP-OCRv4, Tesseract) is used to extract the text content within the unit, generating associated data. The associated data is in the form of "BBOX ID-Layout Type-Coordinate Information-Text Content", providing basic data for subsequent relationship analysis and semantic processing.

[0078] Phase 2: Construction of a two-dimensional relationship graph.

[0079] The two-dimensional relationship in this stage refers to the layout relationship and the semantic relationship.

[0080] Step 2.1: Multimodal model identification of layout logic relationships:

[0081] Input the associated data generated in step 1.3 into a document-specific multimodal model (such as LayoutLMv3 or DocLlama). The model analyzes the logical relationships between each layout unit by integrating the three modal information of text content, BBOX coordinates, and layout type, and constructs a layout relationship diagram.

[0082] The specific implementation process includes:

[0083] (1) The model learns features of the spatial position of the BBOX on the page (such as vertical adjacency, containment, and same width alignment) and the type hierarchy (such as the main text immediately following the first-level heading);

[0084] (2) Output the layout relation triples in the following format: [Source BBOX ID]-[Relation Type]-[Target BBOX ID]. The relation type must include at least the following categories:

[0085] Subordinate, such as title-body text;

[0086] Parallel structures, such as headings at the same level;

[0087] Corresponding, for example, image - caption;

[0088] Includes, for example, a table or cell.

[0089] Step 2.2, Semantic Segmentation and Text Processing:

[0090] Semantic segmentation is performed based on the layout unit division results in step 1.2 to avoid disrupting the layout logic. Independent layout units (such as a single text paragraph or a single table) are prioritized as the basic semantic units. If the unit text is too long (such as exceeding 512 tokens), it is further segmented within the unit according to semantic integrity (such as sentence boundaries) to generate the final text slice Chunk. This ensures that each Chunk contains complete semantics and is associated with the original layout unit information (marking the BBOXID to which the Chunk belongs).

[0091] Step 2.3: Construct a semantic relationship graph:

[0092] For the text slices generated in step 2.2, a semantic relationship graph is constructed using the core process of the GraphRAG graph augmentation retrieval generation system:

[0093] (1) Perform entity recognition (e.g., named entity recognition NER) and relation extraction (e.g., zero-shot relation extraction based on Prompt) on each Chunk to extract entities / concepts (e.g., “artificial intelligence”, “deep learning”) and semantic associations (e.g., “contains”, “applies to”);

[0094] (2) Construct a semantic relationship graph with entities / concepts as nodes and semantic relationships as edges. The node attributes include entity type and associated Chunk ID, and the edge attributes are semantic relationship types.

[0095] (3) The semantic relationship graph and the layout relationship graph are associated through “Chunk ID-BBOX ID” to realize cross-dimensional mapping between the two graphs.

[0096] Step 2.4, Community Testing:

[0097] Using a community clustering algorithm based on the modularity index (Q), such as the Louvain algorithm, communities are divided into typographic and semantic relationship graphs. A summary is generated for each community, which records the meta-information of all nodes and edges within the community.

[0098] Step 2.5, Information Storage:

[0099] Use a vector database (such as ElasticSearch) to record the vector information of the original text of the slice, and use a graph database to record the nodes, edges, confidence, community and other related information of the two relational graphs to form a complete knowledge base for subsequent retrieval.

[0100] In this embodiment, target recognition technology is used to analyze the layout structure of the input document (such as PDF, PPT), dividing the document into different layout units such as title blocks, body text blocks, table blocks, and image blocks; then, a multimodal model is used to analyze the logical relationships between each layout unit (such as subordinate, parallel, corresponding, etc.); when constructing the knowledge base, not only are the vector information and semantic associations of the text slices stored, but the structural relationships of each layout unit are also recorded, forming a multi-dimensional relationship diagram that integrates text semantics and layout logic.

[0101] Example 3:

[0102] This embodiment is a further detailed explanation of the text recall process in step S3 of embodiment 1.

[0103] like Figure 3 As shown, it specifically includes:

[0104] Multi-dimensional information fusion recall

[0105] Step 3.1, User Query Preprocessing:

[0106] The system receives user queries and performs semantic parsing, including extracting core entities from the query and identifying intent by determining the type of user request, such as factual queries or logical reasoning. The query is then converted into vector form, which can be done using the same vector model as text slicing, such as Sentence-BERT.

[0107] Step 3.2, Initial Screening:

[0108] This step primarily involves vector similarity recall. It calculates the vector similarity (cosine similarity in this example) between the query vector and all text slice vectors in the knowledge base, recalling the Top-N (for example, N=50) candidate chunks with the highest similarity. Simultaneously, it retrieves semantic communities based on the query entity, calculates the semantic community relevance, and recalls the X (e.g., X=5) semantic communities with the highest relevance and their contained chunks. It also retrieves layout communities based on the implicit layout requirements of the query (e.g., "content under a certain title"), calculates the layout community relevance, and recalls Y (e.g., Y=5) layout communities and their contained chunks. These are then merged to form a candidate chunk pool, serving as the basic recall result.

[0109] Step 3.3, Refinement:

[0110] The specific values ​​set in this step are all user-defined hyperparameters. For ease of explanation, this embodiment uses direct assignment of values.

[0111] This process includes the following stages:

[0112] (1) Cross-validation phase:

[0113] Perform dual-graph cross-validation on the candidate Chunk pool. If a Chunk exists in both the semantic community recall results and the typography community recall results, it is marked as a "dual-graph cross-validation Chunk", and its base weight is increased by 50%. Then check the mapping consistency between the semantic community and the typography community to which the Chunk belongs. If the two topics match (e.g., the semantic community topic is "deep learning model" and the typography community topic is "section 3.2 model architecture"), the weight is further increased by 10%-20%, thus obtaining the value of "cross-validation bonus".

[0114] (2) Dynamic weight ranking stage:

[0115] The weighting of layout and semantic relationships is dynamically adjusted based on graph complexity metrics. A simple approach could be to use the number of nodes or communities in the graph. This embodiment assigns higher weights to more complex graphs for the following intuitive reasons: If the layout graph is more complex, meaning it has more nodes, it can be understood that the original document corresponding to this graph has a more fragmented layout (such as a PPT that only lists key points). In this case, many logical relationships are implied in the layout rather than in the explanatory text. Conversely, if the semantic graph is more complex, it can be understood that the original document is more inclined towards a neat layout and explanations through text (such as an academic paper). Such documents need to pay more attention to semantic relevance.

[0116] For the reasons mentioned above, the weight allocation rule is as follows:

[0117] If the complexity of the layout diagram / the complexity of the semantic diagram > 1.2 (e.g., in a PPT document), set the layout weight to 0.6 and the semantic weight to 0.4.

[0118] If the ratio is less than 0.8 (e.g., for academic papers), the semantic weight is set to 0.6 and the exclusion weight is set to 0.4.

[0119] Otherwise, both weights are set to 0.5.

[0120] (3) Overall Score Calculation: This step uses the weight values ​​obtained in the previous steps to calculate the overall score of the text slice Chunk. The formula is:

[0121] Overall score = (vector similarity × 0.3) + (semantic community relevance × semantic weight) + (typography community relevance × typography weight) + (cross-validation bonus);

[0122] The top 10 text slices were selected as the final recall results, sorted in descending order of their overall scores.

[0123] Step 3.4, Output the results:

[0124] The Chunk from the final recall results is used as supplementary context and input along with the user's query into the large language model to generate accurate and comprehensive answers.

[0125] The process described in this embodiment allows for simultaneous consideration of semantic similarity and layout logic during text retrieval, thereby significantly improving retrieval effectiveness.

[0126] Example 4:

[0127] This embodiment proposes a device to improve the recall effect of RAG, such as... Figure 4 As shown, it includes:

[0128] Document Input and Layout Structure Parsing Module: Identifies layout units in the input document and extracts the text content within the layout units, generating correlation data between layout and content;

[0129] Two-dimensional relationship graph construction module: performs semantic segmentation based on layout units, and extracts the logical relationships of layout units based on the associated data; constructs a two-dimensional relationship graph of semantic relationships and layout relationships; divides semantic communities and layout communities and stores information;

[0130] Multi-dimensional information fusion recall module: Receives user queries and performs semantic parsing, recalls similar semantic slices from semantic and typesetting communities, performs dual-graph cross-validation, dynamically adjusts weights, calculates and obtains the final recall results.

[0131] The document input and layout structure parsing module includes:

[0132] Multi-format document input unit: Receives unstructured documents uploaded by users, supporting multiple document formats including PDF, PPT, and Word;

[0133] Layout Unit BBOX Recognition Unit: The document layout analysis model based on object detection is used to perform page-level parsing of the input document, identify and output each layout unit;

[0134] OCR Text Extraction Unit: For each identified layout unit, a high-precision OCR model is called to extract the text content within the unit, generating associated data including BBOX ID, layout type, coordinate information, and text content, providing basic data for subsequent relationship analysis and semantic processing.

[0135] The BBOX recognition unit for typesetting includes:

[0136] Locate all visually independent content areas within the page and output the pixel coordinates of each area;

[0137] The document content is structurally broken down into layout units by classifying each area using category tags, including first-level headings, second-level headings, body paragraphs, tables, images, figure captions, and formulas.

[0138] Furthermore, the two-dimensional relationship graph construction module includes:

[0139] Modal model identification of layout logical relationship units: The associated data is input into the document multimodal model. By integrating the three modal information of text content, BBOX coordinates and layout type, the logical relationship between each layout unit is analyzed and a layout relationship diagram is constructed.

[0140] Semantic segmentation and text processing unit: Semantic segmentation is performed based on the typesetting unit. The independent typesetting unit is used as the basic semantic unit. If the text length of the typesetting unit exceeds the set threshold, a second segmentation is performed according to semantic integrity to generate the final text slice. Each text slice contains complete semantics and is associated with the original typesetting unit information BBOX ID.

[0141] Constructing a semantic relationship graph unit: For the text slices, the core process of the graph-enhanced retrieval generation system GraphRAG is used to construct a semantic relationship graph, and a two-dimensional relationship graph is constructed by combining it with the layout relationship graph;

[0142] Community detection unit: Using a community clustering algorithm based on modular exponent Q, communities are divided into layout relationship graph and semantic relationship graph respectively. A summary is generated for layout community and semantic community respectively, and a summary of the meta-information of all nodes and edges in the community is recorded.

[0143] Information storage unit: A vector database is used to record the vector information of the original text of the text slice, and a graph database is used to record the relevant information of the two-dimensional relationship graph, forming a knowledge base for subsequent retrieval.

[0144] Modal model recognition layout logic relationship unit includes:

[0145] Feature learning is performed on the spatial location and type hierarchy of BBOX within the page;

[0146] Output the layout relation triples in the following format: source BBOX ID-relation type-target BBOX ID.

[0147] The construction of semantic relation graph units includes:

[0148] Entity recognition and relation extraction are performed on each text slice to extract entities / concepts and semantic associations;

[0149] Construct a semantic relationship graph with entities / concepts as nodes and semantic relationships as edges. Node attributes include entity type and associated Chunk ID, and edge attributes are semantic relationship types.

[0150] The semantic relationship graph and the layout relationship graph are linked by “Chunk ID-BBOX ID” to achieve the construction of a two-dimensional relationship graph.

[0151] The multi-dimensional information fusion recall module includes:

[0152] User Query Preprocessing Unit: Receives user queries, performs semantic parsing on them, including entity extraction and intent recognition, and converts the queries into vector form;

[0153] Initial screening unit: Vector similarity recall, calculates the vector similarity between the query vector and all text slice vectors in the knowledge base, and recalls the top-N candidate text slices with the highest similarity; simultaneously, based on the query entity, it retrieves semantic communities, calculates the semantic community association degree between the query and the semantic community, and recalls the X semantic communities with the highest association degree and their contained text slices; combined with the implicit typesetting requirements of the query, it retrieves typesetting communities, calculates the typesetting community association degree between the query and the typesetting community, and recalls Y typesetting communities and their contained text slices; these are merged to form a candidate Chunk pool as the basic recall result;

[0154] The fine-tuning unit includes the following stages: Cross-validation stage: Performing dual-graph cross-validation on the candidate Chunk pool to obtain the cross-validation bonus value; Dynamic weight ranking stage: Dynamically adjusting the weight ratio of layout and semantic relationship based on graph complexity index to obtain semantic weight and layout weight; Comprehensive score calculation stage: Calculating the comprehensive score of text slices, comprehensive score = (vector similarity × 0.3) + (semantic community relevance × semantic weight) + (layout community relevance × layout weight) + (cross-validation bonus), and taking the top few scores in descending order as the final recall results;

[0155] Output Unit: The final recall results of the text slice are used as supplementary context and input into the large language model along with the user query to generate a response.

[0156] In the fine-mapping unit, the dual-graph cross-validation includes: if a text slice exists in both the semantic community recall results and the typesetting community recall results, it is marked as a dual-graph cross-validation chunk and a cross-validation bonus value is set; then the mapping consistency between the semantic community and the typesetting community to which the text slice belongs is checked, and if the two themes match, the cross-validation bonus value is further increased.

[0157] In the fine-matrix unit, the dynamic adjustment of the weight ratio between layout and semantic relationships based on graph complexity index includes: if the complexity of the layout relationship graph is greater than the complexity of the semantic relationship graph, the layout weight is higher than the semantic weight; if the complexity of the layout relationship graph is less than the complexity of the semantic relationship graph, the semantic weight is higher than the layout weight; otherwise, the two weights are equal.

[0158] The apparatus for improving RAG recall effect proposed in this embodiment can achieve the method for improving RAG recall effect in any of the embodiments 1, 2, and 3, and has the same technical effect as embodiments 1, 2, and 3.

[0159] The above-described embodiments are merely preferred embodiments of the present invention and are only used to help understand the method and core ideas of this application. The scope of protection of the present invention is not limited to the above embodiments, and all technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A method for improving the recall effect of RAG (Recall Aid Group), characterized in that, include: S1. Document Input and Layout Structure Analysis: Identify layout units in the input document and extract the text content within the layout units to generate correlation data between layout and content; S2. Two-dimensional relationship graph construction: semantic segmentation based on layout units, and extraction of logical relationships between layout units based on the associated data; Construct a two-dimensional relationship graph of semantic and typographic relationships; divide semantic and typographic communities and store information; including: S201. Modal model identification of layout logic relationship: Input the associated data into the document multimodal model, and analyze the logical relationship between each layout unit by integrating the three modal information of text content, BBOX coordinates and layout type, and construct a layout relationship diagram; S202, Semantic Segmentation and Text Processing: Semantic segmentation is performed based on the typesetting unit. The independent typesetting unit is used as the basic semantic unit. If the text length of the typesetting unit exceeds the set threshold, a second segmentation is performed according to semantic integrity to generate the final text slice. Each text slice contains complete semantics and is associated with the original typesetting unit information BBOX ID. S203. Construct a semantic relationship graph: For the text slices, construct a semantic relationship graph using the core process of the graph-enhanced retrieval generation system GraphRAG, and combine it with the layout relationship graph to achieve the construction of a two-dimensional relationship graph; S204, Community Detection: Using a community clustering algorithm based on modular exponent Q, communities are divided into layout relationship graph and semantic relationship graph respectively. A summary is generated for layout community and semantic community respectively, and a summary of the meta-information of all nodes and edges in the community is recorded. S205. Information storage: Use a vector database to record the vector information of the original text of the text slice, and use a graph database to record the relevant information of the two-dimensional relationship graph to form a knowledge base for subsequent retrieval. S3. Multi-dimensional Information Fusion Recall: Receives user queries and performs semantic parsing, recalls similar semantic slices from semantic and typography communities, performs dual-graph cross-validation, dynamically adjusts weights, and calculates and obtains the final recall results; including: S301. User Query Preprocessing: Receive user query, perform semantic parsing on it, including entity extraction and intent recognition, and convert the query into vector form. S302, Initial Screening: Vector Similarity Recall. Calculate the vector similarity between the Query vector and all text slice vectors in the knowledge base, and recall the Top-N candidate text slices with the highest similarity. Simultaneously, retrieve semantic communities based on the Query entity, calculate the semantic community association degree between the Query and the semantic community, and recall the X semantic communities with the highest association degree and their contained text slices. Combine the implicit layout requirements of the Query to retrieve layout communities, calculate the layout community association degree between the Query and the layout community, and recall Y layout communities and their contained text slices. Merge these to form a candidate Chunk pool as the basic recall result. S303, Fine-tuning: Includes the following stages: Cross-validation phase: Perform dual-graph cross-validation on the candidate Chunk pool to obtain cross-validation bonus values; the dual-graph cross-validation includes: if a text slice exists in both the semantic community recall results and the typesetting community recall results, mark it as a dual-graph cross-validation Chunk and set a cross-validation bonus value; then check the mapping consistency between the semantic community and the typesetting community to which the text slice belongs, and if the two topic directions match, further increase the cross-validation bonus value; Dynamic weight ranking stage: Based on the graph complexity index, the weight ratio of layout and semantic relationship is dynamically adjusted to obtain semantic weight and layout weight; Comprehensive score calculation stage: Calculate the comprehensive score of the text slices. Comprehensive score = (vector similarity × 0.3) + (semantic community relevance × semantic weight) + (typography community relevance × typography weight) + (cross-validation bonus). The top few scores in descending order are taken as the final recall results. S304. Output Results: The final recall results of the text slices are used as supplementary context and input into the large language model along with the user query to generate a response.

2. The method for improving RAG recall effectiveness according to claim 1, characterized in that, Step S1 includes: S101. Multi-format document input: Receives unstructured documents uploaded by users, supporting multiple document formats including PDF, PPT, and Word. S102, BBOX Recognition: The document layout analysis model based on object detection is used to perform page-level parsing of the input document, identify and output each layout unit; S103, OCR Text Extraction: For each identified layout unit, a high-precision OCR model is called to extract the text content within the unit, generating associated data including BBOX ID, layout type, coordinate information, and text content, providing basic data for subsequent relationship analysis and semantic processing.

3. The method for improving RAG recall effectiveness according to claim 2, characterized in that, Step S102 specifically includes: S102-1. Locate all visually independent content areas within the page and output the pixel coordinates of each area; S102-2. Classify each area by type. Use classification tags, including first-level headings, second-level headings, body paragraphs, tables, pictures, figure captions, and formulas, to achieve structured decomposition of document content and obtain layout units.

4. The method for improving RAG recall effectiveness according to claim 1, characterized in that, Step S201 specifically includes: S201-1, Perform feature learning on the spatial location and type hierarchy of BBOX within the page; S201-2. Output layout relation triples in the following format: source BBOX ID - relation type - target BBOX ID.

5. The method for improving RAG recall effectiveness according to claim 1, characterized in that, Step S203 specifically includes: S203-1. Perform entity recognition and relation extraction on each text slice to extract entities / concepts and semantic associations; S203-2. Construct a semantic relationship graph with entities / concepts as nodes and semantic relationships as edges. Node attributes include entity type and associated Chunk ID, and edge attributes are semantic relationship types. S203-3. Establish a connection between the semantic relationship diagram and the layout relationship diagram through "Chunk ID-BBOX ID" to realize the construction of a two-dimensional relationship diagram.

6. The method for improving RAG recall effectiveness according to claim 1, characterized in that, In step S303, the dynamic adjustment of the weight ratio of layout and semantic relationship based on graph complexity index includes: if the complexity of the layout relationship graph is greater than the complexity of the semantic relationship graph, the layout weight is higher than the semantic weight; if the complexity of the layout relationship graph is less than the complexity of the semantic relationship graph, the semantic weight is higher than the layout weight; otherwise, the two weights are equal.

7. An apparatus for improving the recall effect of RAG (Recall Agencies), characterized in that, include: Document Input and Layout Structure Parsing Module: Identifies layout units in the input document and extracts the text content within the layout units, generating correlation data between layout and content; Two-dimensional relationship graph construction module: performs semantic segmentation based on layout units, and extracts the logical relationships of layout units based on the associated data; constructs a two-dimensional relationship graph of semantic relationships and layout relationships; divides semantic communities and layout communities and stores information; Multi-dimensional information fusion recall module: Receives user queries and performs semantic parsing, recalls similar semantic slices from semantic and typesetting communities, performs dual-graph cross-validation, dynamically adjusts weights, calculates and obtains the final recall results; The two-dimensional relationship graph construction module includes: Modal model identification of layout logical relationship units: The associated data is input into the document multimodal model. By integrating the three modal information of text content, BBOX coordinates and layout type, the logical relationship between each layout unit is analyzed and a layout relationship diagram is constructed. Semantic segmentation and text processing unit: Semantic segmentation is performed based on the typesetting unit. The independent typesetting unit is used as the basic semantic unit. If the text length of the typesetting unit exceeds the set threshold, a second segmentation is performed according to semantic integrity to generate the final text slice. Each text slice contains complete semantics and is associated with the original typesetting unit information BBOX ID. Constructing a semantic relationship graph unit: For the text slices, the core process of the graph-enhanced retrieval generation system GraphRAG is used to construct a semantic relationship graph, and a two-dimensional relationship graph is constructed by combining it with the layout relationship graph; Community detection unit: Using a community clustering algorithm based on modular exponent Q, communities are divided into layout relationship graph and semantic relationship graph respectively. A summary is generated for layout community and semantic community respectively, and a summary of the meta-information of all nodes and edges in the community is recorded. Information storage unit: A vector database is used to record the vector information of the original text of the text slice, and a graph database is used to record the relevant information of the two-dimensional relationship graph, forming a knowledge base for subsequent retrieval; The multi-dimensional information fusion recall module includes: a user query preprocessing unit: receiving user query queries, performing semantic parsing on them, including entity extraction and intent recognition, and converting the query into vector form; Initial screening unit: Vector similarity recall, calculates the vector similarity between the query vector and all text slice vectors in the knowledge base, and recalls the top-N candidate text slices with the highest similarity; simultaneously, based on the query entity, it retrieves semantic communities, calculates the semantic community association degree between the query and the semantic community, and recalls the X semantic communities with the highest association degree and their contained text slices; combined with the implicit typesetting requirements of the query, it retrieves typesetting communities, calculates the typesetting community association degree between the query and the typesetting community, and recalls Y typesetting communities and their contained text slices; these are merged to form a candidate Chunk pool as the basic recall result; The fine-tuning unit includes the following stages: Cross-validation phase: Perform dual-graph cross-validation on the candidate Chunk pool to obtain the cross-validation bonus value; dual-graph cross-validation includes: if a text slice exists in both the semantic community recall results and the typesetting community recall results, mark it as a dual-graph cross-validation Chunk and set the cross-validation bonus value; then check the mapping consistency between the semantic community and the typesetting community to which the text slice belongs. If the topic directions of the two match, further increase the cross-validation bonus value; Dynamic weight ranking stage: Based on the graph complexity index, the weight ratio of layout and semantic relationship is dynamically adjusted to obtain semantic weight and layout weight; Comprehensive score calculation stage: Calculate the comprehensive score of the text slices. Comprehensive score = (vector similarity × 0.3) + (semantic community relevance × semantic weight) + (typography community relevance × typography weight) + (cross-validation bonus). The top few scores in descending order are taken as the final recall results. Output Unit: The final recall results of the text slice are used as supplementary context and input into the large language model along with the user query to generate a response.

Images