Hybrid rag intelligent retrieval method for architectural drawings and complex specification documents

Abstract

The application belongs to the technical field of retrieval enhancement generation and building document intelligent analysis, and discloses a hybrid RAG intelligent retrieval method for building drawings and complex specification documents, which comprises the following steps: step one, receiving a building PDF document processing request, the request at least including a file address, an original file identification, a vector set name, an object storage configuration and a result notification topic. Through the intelligent slicing algorithm adapted to the building heterogeneous document, a full-page overview and a semantic local slice are generated, the integrity of the table, formula and article context is ensured by combining the four-way expansion completion algorithm, and then the visual dense multi-vector and the text sparse vector are jointly written into the vector database through hybrid vectorization, so that the layout visual recall, the professional term accurate recall and the structured load filtering are simultaneously supported, and the retrieval integrity and the result traceability capability of the building specification articles, table data and drawing labels are significantly improved.

Description

Technical Field

[0001] This invention belongs to the field of search enhancement generation and intelligent parsing of architectural documents, specifically a hybrid RAG intelligent retrieval method for architectural drawings and complex specification documents. Background Technology

[0002] Retrieval Enhancement Generation (RAG) technology typically includes steps such as document parsing, text segmentation, vectorization and database entry, similarity retrieval, context assembly, and large language model generation. Existing common solutions often use text extraction tools to extract text from PDFs, then segment the text according to fixed length, fixed paragraph, or title rules, and then use a text vector model to generate vectors and write them into a vector database. When a user asks a question, the system vectorizes the question and performs similarity retrieval, and uses the retrieved text as the basis for generating the answer using a large language model.

[0003] This type of solution is effective for ordinary text documents, web pages, and office documents with relatively simple structures, but it has obvious limitations for professional PDF documents in the construction industry. Building code documents are not simple natural language text, but are composed of a mixture of clause numbers, hierarchical headings, tables, formulas, symbol explanations, clause annotations, appendices, and partial illustrations. Building drawing documents further include large-format drawings, dimensions, symbols, legends, partial details, and spatial layout relationships. Relying solely on text extraction and fixed-length blocks makes it difficult to fully express the above complex structure.

[0004] Currently, document retrieval in the construction field mainly includes three methods: first, keyword retrieval, which relies on filenames, clause numbers, or full-text indexes; second, OCR plus text vector retrieval, which first recognizes the text in PDFs or scanned documents and then creates text vectors; and third, image-level retrieval based on general visual models or multimodal models. Each of these solutions has its shortcomings. Keyword retrieval struggles to handle semantic similarity issues. For example, if a user enters "applicable conditions for underground cooling," it may not directly match the equivalent professional expression in the code clauses. OCR plus text vector retrieval is effective for the main text, but it is prone to losing structural relationships in cross-page tables, formula contexts, symbol explanations, and partial annotations on drawings. Although general multimodal image retrieval can preserve visual form, if it is not adapted to the structure of the construction document through slicing and completion, it is prone to problems such as missing local information, truncated slice boundaries, incomplete tables, and formulas separated from explanatory text.

[0005] Existing general-purpose RAG technology has at least the following drawbacks: First, fixed text blocks cannot maintain the hierarchical structure of building codes. The chapters, sections, articles, clauses, and items in building codes have strong binding relationships. If they are divided according to fixed lengths, it is easy to separate the article number, main text, annotations, and applicable conditions, resulting in incomplete recall context. Second, the ability to parse table data is insufficient. Building standards contain various tables such as parameter tables, construction requirement tables, and calculation condition tables. Table headers, units, notes, and cross-row and cross-column relationships are part of the retrieval semantics. General text extraction often loses the table topology. Third, the architectural formula retrieval fails. Formulas are usually composed of variables, units, constraints, scope of application and text descriptions. Simple OCR is prone to errors, and fixed slices may separate the formula from the variable interpretation. Fourth, local annotations on large-format architectural drawings are easily truncated. If sliced ​​according to a fixed grid, edge text, dimension lines, symbols, and legends are easily distributed across slices; if only the whole drawing is retained, local details may be weakened when vectorized. Fifth, there is a lack of traceable location mechanism. Existing solutions can often only return text fragments or image links, which cannot accurately indicate which area of ​​the original page or the full image the hit content is located in, which is not conducive to image review, compliance verification and manual review. Sixth, the retrieval vectors are too simple to express. Using only text-dense vectors will weaken the structure of tables, formulas and images; using only visual vectors will make it difficult to accurately hit standard numbers, technical terms and numerical indicators. Summary of the Invention

[0006] The purpose of this invention is to provide a hybrid RAG intelligent retrieval method for architectural drawings and complex specification documents, so as to solve the problems mentioned in the background art.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a hybrid RAG intelligent retrieval method for architectural drawings and complex specification documents, comprising the following steps: Step 1: Receive a request for processing a building PDF document. The request includes at least the file address, original file identifier, vector set name, object storage configuration, and result notification topic. Write the request to a message queue for asynchronous execution. Step 2: Perform idempotency and concurrency control. Construct idempotent keys based on the subscribed topic, collection name, tenant identifier, user identifier, original file identifier, and request random identifier. Record the processing status through Redis to avoid duplicate consumption, and set file-level, page-level, and image-level concurrent semaphores. Step 3: Download and identify the input file type. If it is a PDF, proceed to the PDF page for parsing. If it is an Office file, TXT file, or Markdown file, convert it to PDF first. If it is an image, convert it to a single-page overview image. Step 4: Perform intelligent rendering and slicing on the PDF page. Calculate the dynamic DPI based on the page size to generate a full-page overview image with controlled long sides. For pages that exceed the preset area threshold or meet the complexity triggering conditions, generate a high-definition rendering image and call the visual big model to partition the page layout to obtain candidate boxes for tables, charts, text blocks or local areas. Step 5: Execute the semantic block completion algorithm. First, merge adjacent small blocks for the candidate boxes returned by the visual large model. Then, use a four-way expansion strategy to expand each semantic block to the adjacent semantic blocks or page boundaries. If the visual large model analysis fails, use overlapping grid slicing as a fallback. Step 6: Upload the full-page overview image and slice images to object storage and save the unified path structure; Step 7: Perform multimodal semantic understanding. For full-page overview, table slices, diagram slices, text block slices, and semantic slices, call the visual language model to generate structured fields that include at least a summary and labels. For local slices, use a dual-image input method, with the full-page overview as the context and the local slice as the analysis target. Step 8: Perform hybrid vectorization. For each image, call the visual multimodal vector model to generate dense multivectors. For images with generated semantic fields, concatenate the file name, summary, tag, and dynamic fields into text and call the sparse vector model to generate sparse vectors. Step 9: Write each image unit as a vector point into the vector database. The vector point includes named dense vectors, named sparse vectors, and a payload field. The payload field includes at least the original file identifier, file name, page number, image address, image type, unique image identifier, normalized bounding box, summary, and label. Step 10: Send processing progress and completion notifications so that the upper-layer business can initiate RAG retrieval upon completion.

[0008] Preferably, the intelligent rendering and slicing in step four specifically includes: calculating dynamic DPI based on the page size, where the upper limit of the long side is 4,096 pixels and the DPI is limited to between 72 and 300 pixels; when the page area exceeds a preset area threshold, generating a high-definition rendering image and calling the visual large model to partition the page to obtain candidate boxes, and generating semantic slices after completing the candidate boxes; when the visual large model analysis fails or there are no effective candidate boxes after completion, using overlapping grid slices as a fallback, with a grid slice size of 1,024 pixels and an overlap length of 200 pixels.

[0009] Preferably, the semantic block completion algorithm in step five includes: sorting the candidate boxes returned by the visual large model according to their vertical coordinates; merging candidate boxes that are vertically adjacent and have a horizontal overlap ratio of more than 0.5 and a merged height of less than 2,048 pixels; filtering out noise boxes with a width or height of less than 50 pixels; for each candidate box, if it has horizontal projection overlap with other boxes, expanding upwards and downwards to the boundary of the adjacent box; if it has vertical projection overlap, expanding left and right to the boundary of the adjacent box; if there is no adjacent box, expanding to the page boundary.

[0010] Preferably, the hybrid vectorization in step eight specifically includes: generating a visually dense multi-vector for each image unit; for image units of analyzable type, calling a visual language model to generate a summary, tags, and dynamic fields, concatenating the file name, summary, tags, and dynamic fields into text to generate a sparse vector; writing the dense vector and the sparse vector into the same vector point, wherein the payload of the vector point includes at least the original file identifier, file name, page number, image address, image type, normalized bounding box, summary, and tags.

[0011] Preferably, the idempotency control and concurrency control in step two specifically include: constructing an idempotent key containing the subscription topic, set name, tenant identifier, user identifier, original file identifier, and request random identifier; recording the processing status through Redis, if the key does not exist, writing the processing status and setting a lease, if the key value is the completed status, directly confirming consumption, and if it is another processing status, returning to retry; renewing the lease periodically during processing, and writing the completed status after successful processing; and setting file-level, page-level, and image-level concurrent semaphores.

[0012] Preferably, in the multimodal semantic understanding described in step seven, a single-image mode is used to call the visual language model for the full-page overview image, and a dual-image mode is used for the local slices, wherein the first image is the full-page overview image and the second image is the local slice; the model output includes a summary, tags, and dynamic fields; the dynamic fields include at least the article level path, formula variables and units, table titles, and visible fields; all JSON fields output by the model are incorporated into the vector point payload as dynamic fields, and the field values ​​are recursively expanded when constructing sparse vector text.

[0013] Preferably, the vector point ID in step nine adopts a deterministic UUID generation method, specifically: the original file identifier, page number and image suffix are concatenated into a business key, and then a UUID is calculated for the business key as the vector point ID, so that the same image unit has a stable identifier when the same document is parsed repeatedly.

[0014] Preferably, the complexity triggering conditions in step four include: the page contains a formula image placeholder, the horizontal table line density exceeds a preset threshold, the page aspect ratio is greater than 2 or less than 0.5, and the user requests a mandatory semantic slice.

[0015] Preferably, the method further includes performing a hybrid recall on the vector database during RAG retrieval, and rearranging the results according to the original file identifier, page number, image type, abstract, and tags, and returning the hit image address, abstract, tags, page number, and normalized bounding box to the generative model.

[0016] The beneficial effects of this invention are as follows: Compared with existing general RAG technologies, this method generates full-page overviews and semantic local slices through an intelligent slicing algorithm adapted to heterogeneous architectural documents. It combines a four-way expansion completion algorithm to ensure the integrity of tables, formulas, and clause contexts. Furthermore, it uses hybrid vectorization to jointly write visually dense multi-vectors and textual sparse vectors into a vector database, thereby simultaneously supporting visual recall of layout, precise recall of professional terms, and structured load filtering. This significantly improves the retrieval completeness and traceability of results for architectural code clauses, table data, and drawing annotations. Attached Figure Description

[0017] Figure 1 This is a flowchart of the method steps of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] like Figure 1 As shown, this embodiment of the invention provides a hybrid RAG intelligent retrieval method for architectural drawings and complex specification documents, including the following steps: Step 1: Receive the architectural PDF document processing request. The request should include at least the file address, original file identifier, vector set name, object storage configuration, and result notification topic. Write the request to the message queue for asynchronous execution. Step 2: Perform idempotency and concurrency control. Construct idempotent keys based on the subscribed topic, collection name, tenant identifier, user identifier, original file identifier, and request random identifier. Record the processing status through Redis to avoid duplicate consumption, and set file-level, page-level, and image-level concurrent semaphores. Step 3: Download and identify the input file type. If it is a PDF, proceed to the PDF page for parsing. If it is an Office file, TXT file, or Markdown file, convert it to PDF first. If it is an image, convert it to a single-page overview image. Step 4: Perform intelligent rendering and slicing on the PDF page. Calculate the dynamic DPI based on the page size to generate a full-page overview image with controlled long sides. For pages that exceed the preset area threshold or meet the complexity triggering conditions, generate a high-definition rendering image and call the visual big model to partition the page layout to obtain candidate boxes for tables, charts, text blocks or local areas. Step 5: Execute the semantic block completion algorithm. First, merge adjacent small blocks for the candidate boxes returned by the visual large model. Then, use a four-way expansion strategy to expand each semantic block to the adjacent semantic blocks or page boundaries. If the visual large model analysis fails, use overlapping grid slicing as a fallback. Step 6: Upload the full-page overview image and slice images to object storage and save the unified path structure; Step 7: Perform multimodal semantic understanding. For full-page overview, table slices, diagram slices, text block slices, and semantic slices, call the visual language model to generate structured fields that include at least a summary and labels. For local slices, use a dual-image input method, with the full-page overview as the context and the local slice as the analysis target. Step 8: Perform hybrid vectorization. For each image, call the visual multimodal vector model to generate dense multivectors. For images with generated semantic fields, concatenate the file name, summary, tag, and dynamic fields into text and call the sparse vector model to generate sparse vectors. Step 9: Write each image unit as a vector point into the vector database. The vector point includes named dense vectors, named sparse vectors, and a payload field. The payload field includes at least the original file identifier, file name, page number, image address, image type, image unique identifier, normalized bounding box, summary, and label. Step 10: Send processing progress and completion notifications so that the upper-layer business can initiate RAG retrieval upon completion.

[0020] The intelligent rendering and slicing in step four specifically employ the following algorithms: Algorithm 1: BuildingDocumentSmartSlicing Input: PDF page number (page_i), page width (W_pt), height (H_pt). The maximum length of the MAX_DIM value is 4096. Large format area threshold AREA_T = 2,500,000 High-definition rendering scaling factor SCALE=2.0 Slice size TILE_SIZE=1024, Overlap length OVERLAP=200 Output: Set of image units Images = {overview, semantic_slices or grid_tiles} 1. area ← W_pt × H_pt 2. maxDim ← max(W_pt, H_pt) 3. targetDpi ← (MAX_DIM / maxDim) × 72 4. targetDpi ← clamp(targetDpi, 72, 300) 5. overview ← render(page_i, targetDpi) 6. Images.add(type="overview", image=overview) 7. if area ≤ AREA_T: return Images 8. highRes ← render(page_i, SCALE) 9. upload(overview) 10. blocks ← VLMLayoutAnalyze(overview) 11.blocks← ParseAndScale(blocks,highRes.width, highRes.height) 12.completedBlocks ← CompleteSlicing(blocks, highRes.width,highRes.height) 13. if completedBlocks is not empty: for each block in completedBlocks: slice ← crop(highRes, block) bbox←Normalize(block,highRes.width, highRes.height) Images.add(type=block.label or "slice", image=slice, bbox=bbox) else: step ← TILE_SIZE - OVERLAP for y from 0 to highRes.height step step: for x from 0 to highRes.width step step: w ← min(TILE_SIZE, highRes.width - x) h ← min(TILE_SIZE, highRes.height - y) if edge fragment too small then continue tile ← crop(highRes, x, y, w, h) bbox ← [x / W, y / H, (x+w) / W, (y+h) / H] Images.add(type="tile", image=tile, bbox=bbox) 14. Return Images The algorithm described above uses page geometry and visual layout semantics as the basis for segmentation, while retaining both full-page overview and local slices. This allows tables, formulas, diagrams, and partial annotations in building codes to be used as independent retrieval units, avoiding the structural disintegration problem caused by fixed-length text blocks.

[0021] The semantic block completion algorithm in step five is as follows: Algorithm 2: CompleteSlicingByFourWayExpansion Input: The AI ​​candidate box set B = {b1...bn}, Page width in pixels W, page height in pixels H, Merging gap threshold GAP=100 Maximum merge height MERGE_H=2048 Output: The completed semantic block set C 1. Sort B by y1 2. C ← empty list 3. current ← first block 4. for each next in B[2...n]: vGap ← next.y1 - current.y2 xOverlap ←min(current.x2,next.x2) - max(current.x1,next.x1) overlapRatio ← xOverlap / min(current.width,next.width) mergedHeight ← max(current.y2,next.y2) - min(current.y1,next.y1) if vGap < GAP and overlapRatio > 0.5 and mergedHeight < MERGE_H: current ← union(current,next) if next.label == "table": current.label ← "table" else: C.add(current) current ← next 5. C.add(current) 6. Remove blocks whose width < 50 or height < 50 7. snapshots ← copy(C) 8. for each block ci in C with original snapshot si: newY1 ← 0 newY2 ← H for each snapshot sj where j ≠ i: if HorizontalOverlap(si,sj): if sj.y2 ≤ si.y1: newY1 ← max(newY1, sj.y2) if sj.y1 ≥ si.y2: newY2 ← min(newY2, sj.y1) ci.y1 ← newY1 ci.y2 ← newY2 newX1 ← 0 newX2 ← W for each snapshot sj where j ≠ i: if VerticalOverlap(si,sj): if sj.x2 ≤ si.x1: newX1 ← max(newX1, sj.x2) if sj.x1 ≥ si.x2: newX2 ← min(newX2, sj.x1) ci.x1 ← newX1 ci.x2 ← newX2 9. return C By merging adjacent similar regions and expanding in four directions to neighboring blocks or page boundaries, the integrity of table headers, formula variable explanations, and text context is ensured, significantly reducing the truncation of key information caused by visual model boundary offset.

[0022] The hybrid vectorization in step eight specifically employs the following algorithm: Algorithm 3: HybridMultimodalVectorization Input: Image unit: img, filename: fileName, page number: pageNumber. Full page overview URL, collection name Output: Qdrant vector point 1. imageUrl ← uploadToObjectStorage(img.bytes) 2. analyzable ← img.type in {"overview","table","figure","main_body","slice"} 3. if analyzable: if img.type == "overview": llmFields ← VLMAnalyzeSingleImage(imageUrl) else: llmFields ← VLMAnalyzeDualImage(overviewUrl, imageUrl) summary ← llmFields["summary"] tags ← llmFields["tags"] else: llmFields ← empty 4. denseVectors ← ColQwenEmbedding(imageUrl) 5. sparseVector ← null 6. if llmFields is not empty: text ← fileName + "" + Flatten(llmFields) sparseVector ← BGEM3SparseEmbedding(text) 7. payload ← { parentFileId, fileName, pageNumber, imageUrl, type: img.type, imageId: img.imageId, bbox: img.bbox, summary, tags, other dynamic fields } 8. pointId ← UUID(parentFileId + "-p" + pageNumber + "-" +img.fileSuffix) 9. point ← { id: pointId, vectors: { "dense": denseVectors, "sparse": sparseVector if exists }, payload: payload} 10. upsert(collectionName, point) By writing dense visual vectors and sparse text vectors into the same vector point, it supports visual recall of layout, precise recall of professional terms, and structured load filtering, thus solving the problem of incomplete retrieval caused by single vector representation.

[0023] Step two, specifically the idempotency control and concurrency control, includes: idempotencyKey = subscribeTopic + "|" + collectionName + "|" + tenantId + "|" + userId + "|" + parentFileId + "|" + requestNonce The processing status is recorded in Redis: if the key does not exist, write `PROCESSING:token` and set the lease; if the key value is `DONE`, confirm successful consumption directly; if the key value is another `PROCESSING` status, return to retry later. The lease is periodically renewed during processing; `DONE` is written after successful processing; the idempotent key is released when a retryable exception occurs during processing. Simultaneously set up concurrent semaphores at the file level, page level, and image level.

[0024] In step seven, during multimodal semantic understanding, a single-image mode is used to call the visual language model for the full-page overview image, while a dual-image mode is used for local slices, where the first image is the full-page overview image and the second image is a local slice. The model output includes a summary, labels, and dynamic fields. Dynamic fields include at least the article hierarchy path, formula variables and units, table titles, and visible fields. All JSON fields output by the model are incorporated into the vector point payload as dynamic fields, and the field values ​​are recursively expanded when constructing sparse vector text.

[0025] In step nine, the vector point IDs are generated using a deterministic UUID generation method, specifically as follows: pointBusinessKey = parentFileId + "-p" + pageNumber + "-" +fileSuffix pointId = UUID(pointBusinessKey) This method ensures that the same image unit has a stable identifier when the same document is parsed repeatedly, which facilitates overwrite updates and result tracking.

[0026] Compared with existing general RAG technologies, this method generates full-page overviews and semantic local slices through an intelligent slicing algorithm adapted to heterogeneous architectural documents. It combines a four-way expansion completion algorithm to ensure the integrity of tables, formulas, and clause contexts. Furthermore, it uses hybrid vectorization to jointly write visually dense multi-vectors and textual sparse vectors into a vector database, thereby simultaneously supporting visual recall of layout, precise recall of professional terms, and structured load filtering. This significantly improves the retrieval completeness and traceability of results for architectural code clauses, table data, and drawing annotations.

[0027] Specifically, the intelligent rendering and slicing in step four include: calculating dynamic DPI based on the page size, where the upper limit of the long side is 4,096 pixels and the DPI is limited to between 72 and 300 pixels; when the page area exceeds the preset area threshold, generating a high-definition rendering image and calling the visual large model to partition the page to obtain candidate boxes, and generating semantic slices after completing the candidate boxes; when the visual large model analysis fails or there are no effective candidate boxes after completion, using overlapping grid slices as a fallback, with a grid slice size of 1,024 pixels and an overlap length of 200 pixels.

[0028] By using dynamic DPI, the long side of the full-page overview is controlled to be within 4,096 pixels. At the same time, an area threshold of 2.5 million square points is used to distinguish between ordinary pages and large-format drawings. Large-format pages automatically trigger AI semantic slicing to generate candidate boxes for tables, formulas, and body text. Overlapping grid slices are used as a fallback to ensure that details of local annotations and complex table pages in architectural drawings are not lost, solving the problem that fixed-length text blocks cannot adapt to heterogeneous architectural layouts.

[0029] The semantic block completion algorithm in step five includes: sorting the candidate boxes returned by the large visual model according to their vertical coordinates; merging candidate boxes that are vertically adjacent and have a horizontal overlap ratio of more than 0.5 and a merged height of less than 2,048 pixels; filtering out noise boxes with a width or height of less than 50 pixels; for each candidate box, if it has horizontal projection overlap with other boxes, it is extended upwards and downwards to the boundary of the adjacent box; if it has vertical projection overlap, it is extended to the boundary of the adjacent box to the left and right; if there is no adjacent box, it is extended to the page boundary.

[0030] By merging vertically adjacent candidate boxes with high horizontal overlap, filtering out minor noise boxes, and then expanding each semantic block in four directions to neighboring blocks or page boundaries, table headers and bodies, formula bodies and variable explanations, and clause numbers and text can be preserved in the same frame. This significantly reduces semantic truncation caused by visual model boundary offset and improves the retrieval completeness of tables and formula descriptions in building codes.

[0031] Specifically, step eight, hybrid vectorization, includes: generating visually dense multi-vectors for each image unit; for image units of analyzable types, calling a visual language model to generate summaries, labels, and dynamic fields; concatenating the filename, summary, labels, and dynamic fields into text to generate sparse vectors; and writing the dense vectors and sparse vectors into the same vector point, where the vector point's payload includes at least the original file identifier, filename, page number, image address, image type, normalized bounding box, summary, and labels.

[0032] By combining dense visual vectors and sparse text vectors into the same vector point, the retrieval system can simultaneously support visual similarity recall, precise recall of technical terms, and structured load filtering. Table formats, formula images, specification numbers, and parameter values ​​can all be effectively indexed, solving the problem of neglecting one aspect in architectural document retrieval when using a single text vector or a single visual vector.

[0033] Specifically, the idempotency and concurrency control in step two include: constructing an idempotent key containing the subscription topic, collection name, tenant identifier, user identifier, original file identifier, and request random identifier; recording the processing status through Redis, writing a "processing" status and setting a lease if the key value is not found, directly confirming consumption if the key value is complete, and returning to retry if it is any other "processing" status; renewing the lease periodically during processing, and writing a "complete" status after successful processing; and setting file-level, page-level, and image-level concurrent semaphores.

[0034] By using composite idempotent keys and Redis state leases, duplicate consumption of the message queue is avoided, which would lead to the same document being parsed and entered into the database multiple times. Three layers of concurrent semaphores at the file level, page level, and image level provide fine-grained control over resource consumption, preventing system overload when processing large batches of building documents and ensuring the reliability and stability of the database entry process.

[0035] In step seven, the multimodal semantic understanding uses a single-image mode to call the visual language model for the full-page overview image and a dual-image mode for local slices, where the first image is the full-page overview image and the second image is a local slice. The model output includes a summary, labels, and dynamic fields. The dynamic fields include at least the article level path, formula variables and units, table titles, and visible fields. All JSON fields output by the model are incorporated into the vector point payload as dynamic fields, and the field values ​​are recursively expanded when constructing sparse vector text.

[0036] The dual-image mode allows local slices to have the full page's chapter titles, layout, and contextual information when they are understood, enabling accurate identification of the slice's role in the whole page; dynamic fields (article level paths, formula variables and units, table titles, etc.) store professional structured information in the payload and expand it into a sparse vector, enhancing the ability to accurately recall article numbers, parameter values, etc.

[0037] In step nine, the vector point ID is generated using a deterministic UUID generation method. Specifically, the original file identifier, page number, and image suffix are concatenated into a business key, and then a UUID is calculated for this business key as the vector point ID, so that the same image unit has a stable identifier when the same document is parsed repeatedly.

[0038] Deterministic IDs ensure that when the same document is parsed repeatedly, the same image unit (full page overview, same semantic slice, same grid slice) generates the same vector point ID. The vector database performs an overwrite update instead of an append, avoiding the remnants of old version slices, which facilitates multiple version iterations of building codes and incremental maintenance of project drawing libraries.

[0039] The complexity triggering conditions in step four include: the page contains a formula image placeholder, the horizontal table line density exceeds a preset threshold, the page aspect ratio is greater than 2 or less than 0.5, and the user request specifies a forced semantic slice.

[0040] By introducing the aforementioned complexity triggering conditions, even pages with standard dimensions but containing formulas, wide tables, multi-column text, or pages requiring fine segmentation as specified by the user can enter the AI ​​semantic slicing process. This avoids the oversight of complex but small pages due to relying solely on area thresholds, thus improving the comprehensiveness and flexibility of intelligent slicing. The method also includes performing hybrid recall on the vector database during RAG retrieval, and rearranging the results by original file identifier, page number, image type, abstract, and tags, returning the hit image address, abstract, tags, page number, and normalized bounding box to the generative model.

[0041] The multi-dimensional rearrangement after hybrid recall prioritizes the retention of high-quality hits in table, text, and full-page overview types. The returned normalized bounding boxes support the front-end to accurately highlight the hit areas on the full-page overview map, meeting the business needs of architectural drawing review and compliance verification for traceable and verifiable search results.

[0042] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0043] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A hybrid RAG intelligent retrieval method for architectural drawings and complex specification documents, characterized in that, Includes the following steps: Step 1: Receive a request for processing a building PDF document. The request includes at least the file address, original file identifier, vector set name, object storage configuration, and result notification topic. Write the request to a message queue for asynchronous execution. Step 2: Perform idempotency and concurrency control. Construct idempotent keys based on the subscribed topic, collection name, tenant identifier, user identifier, original file identifier, and request random identifier. Record the processing status through Redis to avoid duplicate consumption, and set file-level, page-level, and image-level concurrent semaphores. Step 3: Download and identify the input file type. If it is a PDF, proceed to the PDF page for parsing. If it is an Office file, TXT file, or Markdown file, convert it to PDF first. If it is an image, convert it to a single-page overview image. Step 4: Perform intelligent rendering and slicing on the PDF page. Calculate the dynamic DPI based on the page size to generate a full-page overview image with controlled long sides. For pages that exceed the preset area threshold or meet the complexity triggering conditions, generate a high-definition rendering image and call the visual big model to partition the page layout to obtain candidate boxes for tables, charts, text blocks or local areas. Step 5: Execute the semantic block completion algorithm. First, merge adjacent small blocks for the candidate boxes returned by the visual large model. Then, use a four-way expansion strategy to expand each semantic block to the adjacent semantic blocks or page boundaries. If the visual large model analysis fails, use overlapping grid slicing as a fallback. Step 6: Upload the full-page overview image and slice images to object storage and save the unified path structure; Step 7: Perform multimodal semantic understanding. For full-page overview, table slices, diagram slices, text block slices, and semantic slices, call the visual language model to generate structured fields that include at least a summary and labels. For local slices, use a dual-image input method, with the full-page overview as the context and the local slice as the analysis target. Step 8: Perform hybrid vectorization. For each image, call the visual multimodal vector model to generate dense multivectors. For images with generated semantic fields, concatenate the file name, summary, tag, and dynamic fields into text and call the sparse vector model to generate sparse vectors. Step 9: Write each image unit as a vector point into the vector database. The vector point includes named dense vectors, named sparse vectors, and a payload field. The payload field includes at least the original file identifier, file name, page number, image address, image type, unique image identifier, normalized bounding box, summary, and label. Step 10: Send processing progress and completion notifications so that the upper-layer business can initiate RAG retrieval upon completion.

2. The hybrid RAG intelligent retrieval method for architectural drawings and complex specification documents according to claim 1, characterized in that: The intelligent rendering and slicing in step four specifically include: calculating dynamic DPI based on page size, where the upper limit of the long side is 4,096 pixels and the DPI is limited to between 72 and 300 pixels; when the page area exceeds a preset area threshold, generating a high-definition rendering image and calling the visual large model to partition the page to obtain candidate boxes, and generating semantic slices after completing the candidate boxes; when the visual large model analysis fails or there are no effective candidate boxes after completion, using overlapping grid slices as a fallback, with a grid slice size of 1,024 pixels and an overlap length of 200 pixels.

3. The hybrid RAG intelligent retrieval method for architectural drawings and complex specification documents according to claim 1, characterized in that: The semantic block completion algorithm in step five includes: sorting the candidate boxes returned by the visual large model according to their vertical coordinates; merging candidate boxes that are vertically adjacent and have a horizontal overlap ratio of more than 0.5 and a merged height of less than 2,048 pixels; filtering out noise boxes with a width or height of less than 50 pixels; for each candidate box, if it has horizontal projection overlap with other boxes, it is extended upwards and downwards to the boundary of the adjacent box; if it has vertical projection overlap, it is extended left and right to the boundary of the adjacent box; if there is no adjacent box, it is extended to the page boundary.

4. The hybrid RAG intelligent retrieval method for architectural drawings and complex specification documents according to claim 1, characterized in that: The hybrid vectorization in step eight specifically includes: generating a visually dense multi-vector for each image unit; for image units of analyzable types, calling a visual language model to generate a summary, tags, and dynamic fields, concatenating the file name, summary, tags, and dynamic fields into text to generate a sparse vector; and writing the dense vector and the sparse vector into the same vector point, wherein the payload of the vector point includes at least the original file identifier, file name, page number, image address, image type, normalized bounding box, summary, and tags.

5. The hybrid RAG intelligent retrieval method for architectural drawings and complex specification documents according to claim 1, characterized in that: The idempotency and concurrency control in step two specifically include: constructing an idempotent key containing the subscription topic, set name, tenant identifier, user identifier, original file identifier, and request random identifier; recording the processing status through Redis, if the key does not exist, writing the processing status and setting the lease, if the key value is the completed status, directly confirming consumption, and if it is another processing status, returning to retry; renewing the lease periodically during processing, and writing the completed status after successful processing; and setting file-level, page-level, and image-level concurrent semaphores.

6. The hybrid RAG intelligent retrieval method for architectural drawings and complex specification documents according to claim 1, characterized in that: In the multimodal semantic understanding described in step seven, a single-image mode is used to call the visual language model for the full-page overview image, and a dual-image mode is used for local slices, where the first image is the full-page overview image and the second image is a local slice; the model output includes a summary, tags, and dynamic fields; the dynamic fields include at least the article level path, formula variables and units, table titles, and visible fields; all JSON fields output by the model are incorporated into the vector point payload as dynamic fields, and the field values ​​are recursively expanded when constructing sparse vector text.

7. The hybrid RAG intelligent retrieval method for architectural drawings and complex specification documents according to claim 1, characterized in that: The vector point ID in step nine adopts a deterministic UUID generation method, which is as follows: the original file identifier, page number and image suffix are concatenated into a business key, and then a UUID is calculated for the business key as the vector point ID, so that the same image unit has a stable identifier when the same document is parsed repeatedly.

8. The hybrid RAG intelligent retrieval method for architectural drawings and complex specification documents according to claim 1, characterized in that: The complexity triggering conditions in step four include: the page contains a formula image placeholder, the horizontal table line density exceeds a preset threshold, the page aspect ratio is greater than 2 or less than 0.5, and the user requests a mandatory semantic slice.

9. The hybrid RAG intelligent retrieval method for architectural drawings and complex specification documents according to claim 1, characterized in that: The method further includes performing hybrid recall on the vector database during RAG retrieval, and rearranging the results according to the original file identifier, page number, image type, abstract, and tags, and returning the hit image address, abstract, tags, page number, and normalized bounding box to the generative model.

Images