Intelligent archive processing method and system based on multi-modal large model

Abstract

The application relates to the technical field of artificial intelligence and archive management, and discloses an archive intelligent processing method and system based on a multi-modal large model, which comprises the following steps: preprocessing multi-format archives to generate structured texts, generating classification labels based on the multi-modal large model, and performing key entity extraction to generate metadata labels; splitting the labeled structured texts into semantic text blocks by adopting a semantic-granularity coupling splitting strategy, adding directory level path metadata, and constructing dense semantic vector indexes and sparse keyword vector indexes as double-layer indexes; receiving a user query, performing multi-path hybrid retrieval based on the double-layer indexes to obtain relevant knowledge fragments; generating an answer by using the multi-modal large model, and binding the unique identifier of the source text block and the directory level path metadata for the assertion in the answer. The application solves the problems of poor industry adaptability, insufficient multi-format processing capacity and low intelligentization degree of retrieval in the prior art.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and archival management technology, and in particular to an intelligent archival processing method and system based on a multimodal large model. Background Technology

[0002] Records management is a core component of enterprise information management and knowledge asset protection, encompassing the entire process of data collection, classification, labeling, storage, retrieval, and utilization. With the deepening of enterprise digital transformation, the scale and complexity of archival data are increasing daily, posing significant challenges to traditional records management methods.

[0003] Traditional record management systems primarily rely on manual operation and rule-based database retrieval technology. In the classification stage, record managers typically manually identify and categorize records according to a predetermined classification system; in the annotation stage, manual reading of record contents and manual entry of various metadata fields are required; and in the retrieval stage, traditional database query methods based on keyword matching are commonly used. This processing model, which relies on manual intervention and fixed rules, has the following main shortcomings: 1. Poor industry adaptability. Different industries have their own specific file classification systems and metadata description standards. For example, the drainage industry has established a multi-level classification system based on the "Three-in-One" system, with strict hierarchical and coding requirements for file classification. General classification models have not been deeply adapted to the specific classification standards and professional terminology of this industry, resulting in classification accuracy that is difficult to meet business requirements in practical applications, requiring a large amount of manual review and correction.

[0004] 2. Insufficient multi-format document processing capabilities. Corporate documents typically contain multiple formats, such as scanned documents, PDF documents, engineering drawings, complex tables, and other unstructured data. Traditional solutions mostly focus on plain text processing. For image-based documents such as scanned documents, even with optical character recognition technology, recognition accuracy often declines due to unstable image quality and complex layouts, making it difficult to provide high-quality text input for subsequent classification and annotation.

[0005] 3. Fragmented processing flow and low efficiency. In traditional solutions, preprocessing, classification, and annotation are usually completed separately by independent systems or manually, with a lack of information collaboration between the stages. For example, classification results cannot be automatically used as the basis for selecting metadata templates in the annotation stage, forcing each stage to perform independent contextual understanding, which is not only inefficient but also prone to producing inconsistent results.

[0006] 4. Low level of intelligence in retrieval. Traditional keyword-based retrieval methods cannot understand the semantic intent of users' natural language queries. When users ask questions in natural language, the system struggles to accurately understand the meaning of the question and connect it to relevant document content, resulting in low recall and precision. Furthermore, search results typically only return a list of matching documents, lacking a direct answer to the user's question and a verifiable mechanism for tracing the source of the answer. Users need to read a large number of documents themselves to obtain the information they need.

[0007] Therefore, there is an urgent need for an intelligent archive processing technology solution that can adapt to industry-specific classification standards, effectively process multi-format archives, achieve intelligent collaborative processing throughout the entire process, and provide semantic intelligent retrieval and traceable answer generation. Summary of the Invention

[0008] The purpose of this application is to provide an intelligent archival processing method and system based on a multimodal large model, in order to solve the problems of poor adaptability to the archival processing industry, insufficient multi-format processing capabilities, fragmented processes and low efficiency, low level of intelligent retrieval and lack of traceability mechanism in the existing technology.

[0009] To achieve the aforementioned objectives, the technical solution disclosed in this application includes: The intelligent archival processing method based on a multimodal large model includes the following steps: The input multi-format archives are preprocessed to generate structured text. Based on the multimodal large model, the semantic features of the structured text are mapped to a hierarchical classification label system to generate classification labels. Based on the classification labels, a metadata extraction template is selected to extract key entities and generate metadata labels. The annotated structured text is split into several semantic text blocks using a semantic-granular coupling splitting strategy. Based on the directory structure of the structured text, directory hierarchy path metadata is added, and a dense semantic vector index and a sparse keyword vector index are constructed as a two-level index. Receive user queries and perform multi-path hybrid retrieval based on the dual-layer index to obtain relevant knowledge fragments; Based on the relevant knowledge fragments, the multimodal large model is used to generate answers, and the unique identifiers of the relevant knowledge fragments and the directory hierarchy path metadata are bound to the assertions in the answers.

[0010] Preferably, it also includes: performing compliance verification on the category tags and the metadata tags through a business rules engine, and conducting incremental learning training and iterative optimization based on human feedback.

[0011] Preferably, the multimodal large model is fine-tuned for domain adaptability by injecting industry classification standards and historical archive data.

[0012] Preferably, the preprocessing includes: using an optical character recognition module to perform image denoising, image tilt correction, document layout analysis, and scene text recognition on the multi-format archives.

[0013] Preferably, the semantic-granular coupling splitting strategy includes: calculating the semantic similarity of adjacent text segments, splitting when the similarity is lower than a preset threshold, forcibly splitting when the length of a text block exceeds a preset upper limit, and merging with adjacent blocks when the length is lower than a preset lower limit.

[0014] Preferably, the dense semantic vector index is constructed using a hierarchical navigable small-world graph index, and the sparse keyword vector index is constructed using an inverted index.

[0015] Preferably, the multi-path hybrid retrieval includes: performing retrieval in parallel at the semantic layer and the structural layer, reordering the data through a cross-coding reorderer, and performing referential resolution, entity information completion, and query intent clarification on the user query.

[0016] Preferably, the method further includes: performing deduplication processing on the relevant knowledge fragments, wherein the deduplication processing includes similarity estimation based on minimum hash local sensitive hashing and redundancy recovery based on semantic clustering.

[0017] Preferably, the method further includes: performing context aggregation on the relevant knowledge fragments, wherein the context aggregation includes anchoring chapters based on the directory hierarchy path metadata, supplementing the content of the upper-level chapters along the directory hierarchy, and performing semantic compression on the aggregated context.

[0018] This invention also discloses an intelligent archival processing system based on a multimodal large model, comprising: The preprocessing module is configured to preprocess multi-format archives to generate structured text; The classification and labeling module is configured to use a multimodal large model to classify the structured text and generate classification labels, and select a metadata extraction template based on the classification labels to generate metadata labels. The knowledge base construction module is configured to use a semantic-granular coupling splitting strategy to divide the annotated structured text into blocks and attach directory-level path metadata, and construct a dense semantic vector index and a sparse keyword vector index as a two-layer index. The question-and-answer retrieval module is configured to perform multi-path hybrid retrieval based on the dual-level index to obtain relevant knowledge fragments, generate answers using the multimodal large model, and bind the unique identifier of the relevant knowledge fragments and the directory-level path metadata.

[0019] Beneficial effects This application adopts a serial collaborative processing architecture based on a multimodal large model, which unifies preprocessing, classification, and annotation on the same model base, achieving lossless information transfer between stages. This serial collaborative design avoids the problems of independent processing and repeated understanding of context in traditional solutions, making the overall processing flow both efficient and consistent.

[0020] This application employs a combination of semantic-granular coupling splitting strategy and directory-driven structured splitting strategy in the document segmentation stage. While preserving the chapter-level structure information of the archives, it enables subsequent retrieval to take into account semantic relevance and precise keyword matching, thereby improving the comprehensiveness and accuracy of the retrieval.

[0021] This application performs multi-path hybrid retrieval at the semantic and structural levels in parallel during the retrieval phase, comprehensively considering content relevance and archive structure location. During the answer generation phase, each assertion in the generated answer is bound to a unique identifier of the source text block and directory-level path metadata, achieving precise tracing of the answer to the original text. This effectively improves the verifiability and credibility of the intelligent retrieval results and reduces the risks associated with the large model illusion. Attached Figure Description

[0022] Figure 1 This is a flowchart illustrating a preferred embodiment of the intelligent archive processing method based on a multimodal large model provided by the present invention. Figure 2 This is a schematic diagram of the structure of an intelligent archive processing system based on a multimodal large model provided in a preferred embodiment of the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described below with reference to the accompanying drawings. In the description of this invention, it should be understood that the terms "upper," "lower," "front," "rear," "left," "right," "top," "bottom," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention.

[0024] Example 1 To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to specific embodiments. It should be understood that the specific embodiments described herein are only for explaining this application and are not intended to limit the scope of protection of this application.

[0025] The intelligent archival processing method based on a multimodal large model described in this application can be implemented in a computing system including a computer processor, memory, and communication interface. In one specific embodiment, the system is deployed in an enterprise intranet environment, including a server cluster configured with GPU accelerator cards for running multimodal large model inference tasks; a distributed storage system for storing original archival files, processing results, and vector indexes; and an application server for providing API interfaces for external systems to call.

[0026] The intelligent archival processing system based on a multimodal large model provided in this application adopts a hierarchical modular design, such as... Figure 2 As shown, it includes the following core modules: Preprocessing module: Configured to perform multimodal preprocessing on multi-format archives to generate structured text. This module integrates optical character recognition capabilities and supports unified processing of various archive formats such as scanned documents, PDF documents, and image files.

[0027] Classification and labeling module: configured to use a multimodal large model to classify the structured text and generate classification labels, and dynamically select a metadata extraction template to generate metadata labels based on the classification labels.

[0028] Knowledge base construction module: Configured to use a semantic-granular coupling splitting strategy to divide the annotated structured text into blocks and attach directory-level path metadata, and build a dense semantic vector index and a sparse keyword vector index as a two-level index.

[0029] The question-and-answer retrieval module is configured to perform multi-path hybrid retrieval based on the dual-level index to obtain relevant knowledge fragments, generate answers using the multimodal large model, and bind the unique identifier of the relevant knowledge fragments and the directory-level path metadata.

[0030] like Figure 1 As shown, the intelligent archival processing method based on a multimodal large model provided in this application includes the following steps: S1: Preprocess the input multi-format archives to generate structured text, map the semantic features of the structured text to a hierarchical classification label system based on the multimodal large model to generate classification labels, and select a metadata extraction template according to the classification labels to extract key entities and generate metadata labels.

[0031] S11: Multi-format document preprocessing. For the input multi-format documents (including scanned copies, PDF documents, image files, etc.), preprocessing is first performed to generate structured text. In some preferred embodiments, the preprocessing includes: using an optical character recognition module to perform image denoising, image tilt correction, document layout analysis, and scene text recognition on the multi-format documents.

[0032] For processing scanned documents, existing technologies typically employ general-purpose OCR (Optical Character Recognition) tools for conversion. However, corporate historical archives often suffer from issues such as long storage time, yellowed paper, obvious creases, and skewed scans. General-purpose OCR tools experience a significant drop in accuracy when dealing with these low-quality images and struggle to accurately reproduce complex document layouts (such as multi-column layouts and cross-page tables), resulting in poor-quality input data for subsequent processing. To address these problems, in some preferred embodiments, the preprocessing described in this application employs a combined optimization scheme, including: using an optical character recognition module to perform image denoising, image skew correction, document layout analysis, and scene text recognition on the multi-format archives.

[0033] Specifically, image denoising uses an adaptive filtering algorithm to remove noise and stains from the scanned document; image tilt correction rotates tilted document images to a horizontal position by detecting document edges and text line directions; document layout analysis identifies different elements in the document, such as text areas, table areas, and image areas, and determines their spatial relationships; scene text recognition accurately extracts text from complex backgrounds, supporting the recognition of multiple languages ​​and clear handwriting. After preprocessing, clean, structured text content is output and associated with the source file.

[0034] It should be understood that, in addition to scene text recognition based on multimodal large models, traditional CRNN architecture (CNN+BiLSTM+CTC) or other deep learning OCR models such as PaddleOCR and Tesseract can also be used as alternative implementation methods.

[0035] S12: Automatic classification based on a multimodal large model. The preprocessed structured text is fed into a multimodal large model for semantic understanding. The model analyzes the text content, understands its semantic meaning, and maps semantic features to a pre-defined hierarchical classification label system, outputting structured classification labels.

[0036] When dealing with industry-specific document classification, the conventional approach is to directly use general pre-trained large language models (such as GPT-4, Tongyi Qianwen, etc.) for zero-shot classification. However, while general models possess broad knowledge, they lack in-depth understanding of the professional terminology and classification standards of specific industries (such as drainage, power, and medical), and cannot grasp the specific classification logic within an enterprise. As a result, when faced with industry-specific classification tasks, the classification accuracy is difficult to meet the standards of practical business use, often resulting in "seemingly correct but actually incorrect" errors.

[0037] For example, for a construction project contract, the model identifies that it belongs to the "Construction Contract" subclass under the "Construction Project" category and outputs multi-level category labels such as: first-level category code "JS", second-level category code "24", third-level category code "01", and fourth-level category code "01".

[0038] To address this issue, in some preferred embodiments, the multimodal large model undergoes domain-adaptive fine-tuning by injecting industry classification standards and historical archive data. For example, for the drainage industry, widely accepted industry classification standards (such as the "Three-in-One" system) and metadata rules are injected into the model for fine-tuning, enabling the model to possess accurate industry semantic understanding capabilities and improve classification accuracy. Besides full-parameter fine-tuning, efficient parameter fine-tuning methods such as LoRA and Adapter can be used as alternatives, reducing training costs while maintaining model performance.

[0039] By leveraging a unified capability foundation and domain-adaptive fine-tuning of a multimodal large model, classification accuracy in real-world archival data testing can be significantly improved.

[0040] S13: Automatic labeling based on classification results. The corresponding metadata extraction template is dynamically selected based on the classification tags. Different types of archives have different metadata field requirements. For example, construction contracts require extraction of fields such as the contracting party (Party A), contracting party (Party B), signing date, and contract amount, while equipment maintenance records require extraction of fields such as equipment name, maintenance time, and maintenance content.

[0041] The multimodal large model is used to extract key entities from the structured text, identify and extract target entity information, and populate the corresponding metadata fields to generate metadata tags. For example, from a construction contract, the following can be extracted: "Contractor: Chengdu Drainage Co., Ltd.", "Contractor: XX Construction Group", "Signing Date: May 10, 2023", etc.

[0042] In the result verification stage, traditional methods typically rely entirely on manual review or can only perform format checks based on simple regular expressions. Purely manual review is inefficient and prone to errors due to fatigue; while simple rule checks cannot determine the logical correctness of the content, and the system cannot learn from errors, leading to the recurrence of the same errors, and the model's capabilities cannot improve with use.

[0043] In some preferred embodiments, this application introduces an intelligent closed-loop optimization mechanism, including: performing compliance verification on the classification tags and the metadata tags through a business rule engine, and performing incremental learning training and iterative optimization based on human feedback.

[0044] The business rules engine incorporates industry compliance verification rules to automatically validate classification and labeling results. For example, it verifies whether the encoding format of classification tags conforms to standards and whether required metadata fields are complete. The system supports using manually verified correct results as incremental training data, feeding them back to the model for iterative fine-tuning, forming a self-optimization loop to continuously improve processing accuracy.

[0045] S2: The annotated structured text is split into several semantic text blocks using a semantic-granular coupling splitting strategy. Based on the directory structure of the structured text, directory hierarchy path metadata is added, and a dense semantic vector index and a sparse keyword vector index are constructed as a two-level index. S21: Semantic-granular coupling splitting strategy. The annotated structured text is divided into blocks using a semantic-granular coupling splitting strategy.

[0046] When building vector knowledge bases, existing text segmentation methods mainly employ a fixed-length sliding window strategy (e.g., segmenting into 500-character blocks). This mechanical segmentation often disrupts sentence integrity, resulting in a complete semantic paragraph being split into two different blocks, or irrelevant content being forcibly spliced ​​together. This severely damages the semantic representation of text blocks, leading to inaccurate retrieval or loss of key information during retrieval.

[0047] To overcome the drawbacks of fixed-length segmentation, in some preferred embodiments, this application adopts a semantic-granular coupling segmentation strategy, including: calculating the semantic similarity of adjacent text segments, segmenting when the similarity is lower than a preset threshold, forcibly segmenting when the length of a text block exceeds a preset upper limit, and merging with adjacent blocks when the length is lower than a preset lower limit.

[0048] Specifically, sentence-level semantic boundaries are used as the core segmentation points to ensure thematic coherence of text blocks. Maximum token count constraints (e.g., 512 or 1024 tokens) and minimum overlap constraints are set to suppress vector representation distortion caused by excessive length. Semantic similarity is calculated for adjacent blocks; when the similarity exceeds a preset threshold, highly similar blocks are automatically aggregated into "superblocks," strengthening cross-block contextual associations. This strategy ensures the thematic semantic integrity of each text block, balancing semantic integrity with the characteristics of the archive structure.

[0049] S22: Directory-driven structured splitting. Based on the directory structure of the structured text, append directory hierarchy path metadata to each semantic text block. Parse the document directory to generate a multi-level heading tree, where each tree node stores the heading text, hierarchy relationship, and the index of the pointed-to fragment block.

[0050] Conventional RAG (Retrieval Augmentation) systems typically treat long documents as flat sequences and slice them into chunks, ignoring the document's hierarchical structure (such as chapters and table of contents). This results in retrieved fragments often being isolated pieces of text lacking context (e.g., retrieving "Party A's Responsibilities," but not knowing whether it belongs to the "General Terms" or "Special Terms" section), severely impacting the large model's understanding and judgment of the information.

[0051] In light of this, this application proposes a directory-driven structured decomposition strategy. The system parses the document directory, generating a multi-level heading tree. Each tree node stores the heading text, hierarchical relationship, and the index of the corresponding segment. For example, for a document containing chapters such as "Chapter 1 Project Overview" and "Chapter 2 Construction Plan," the system extracts the directory structure and marks the chapter path to which each text block belongs, such as "Chapter 2 Construction Plan > 2.1 Construction Preparation > 2.1.1 Material Preparation." This structured decomposition method supports quick location of corresponding text through heading paths, enabling bidirectional "chapter-paragraph" retrieval while preserving the document's chapter hierarchy information.

[0052] S23: Construction of a two-level vector index. A dense semantic vector index and a sparse keyword vector index are constructed as a two-level index for the semantic text block.

[0053] Most mainstream vector retrieval systems currently only build indexes based on dense vectors. While dense vectors perform excellently in semantic matching (e.g., matching "apple" and "fruit"), they often perform poorly when handling proper nouns, precise numbers, or low-frequency words (e.g., specific contract numbers, equipment models), easily resulting in "semantically related but detail-incorrect" errors. Traditional keyword indexes (such as Elasticsearch) cannot understand semantics (e.g., they cannot match "computer" and "computer").

[0054] To balance semantic understanding and accurate matching, in some preferred embodiments, this application constructs a two-layer indexing system: the dense semantic vector index is constructed using a hierarchical navigable small-world graph index (HNSW), and the sparse keyword vector index is constructed using an inverted index.

[0055] Specifically, a vector embedding model is used to transform text blocks into dense vectors (capturing semantic information) and sparse vectors (preserving keyword information). The dense vector index is constructed using the HNSW algorithm, supporting efficient approximate nearest neighbor search; the sparse vector index uses an inverted index structure, supporting precise keyword matching queries. Simultaneously, structural context metadata, such as the hierarchical path within the directory tree, is maintained for each text block.

[0056] It should be understood that, in addition to HNSW indexing, other approximate nearest neighbor indexing algorithms such as IVF (Inverted File Index) and PQ (Product Quantization) can be used as alternatives; for sparse vector indexing, in addition to inverted indexing, methods such as BM25 or SPLADE can also be used. Besides the methods described in the text, vector embedding models such as BGE-M3, M3E-large, or other open-source / self-trained embedding models can also be used to generate dense and sparse vectors.

[0057] S3: Receive user queries and perform multi-path hybrid retrieval based on the dual-level index to obtain relevant knowledge fragments. After receiving a user's natural language query, the system performs a multi-path hybrid retrieval based on the two-layer index.

[0058] In some preferred embodiments, the multi-path hybrid retrieval includes: performing retrieval in parallel at the semantic and structural layers, reordering the data using a cross-coding reorderer, and performing referential resolution, entity information completion, and query intent clarification on the user query.

[0059] S31: Question Enhancement. When users engage in multi-turn conversations or ask questions, their queries are often colloquial, vague, or contain pronouns (e.g., "When was it signed?"). If such raw queries are used directly for retrieval, the system cannot determine which contract "it" refers to, resulting in completely irrelevant search results.

[0060] To improve search accuracy, this application introduces a question enhancement step before the search. Question enhancement includes referential resolution (replacing pronouns with explicit entity names), entity information completion (filling in missing entity information based on context), and query intent clarification (clarifying the user's true intent in the query). For example, "When was it signed?" is enhanced to "When was the construction contract for the Xiyuan Water Purification Plant signed?", thereby ensuring the accuracy of the search.

[0061] S32: Multi-path parallel retrieval, performing multi-path recall at the semantic and structural layers in parallel: 1. Semantic layer retrieval: Based on dense semantic vector index, vector similarity retrieval is performed to recall semantically related text blocks; at the same time, based on sparse keyword vector index, keyword matching retrieval is performed to recall text blocks containing keywords.

[0062] 2. Structural layer retrieval: Based on the structural information such as document titles and chapter names mentioned in the query, targeted retrieval is performed through the directory tree structure.

[0063] S33: Reordering and Deduplication. A cross-encoder reorderer is used to reorder the initial search results based on relevance, accurately evaluating the relevance of each candidate text block to the query. It should be understood that, in addition to the RoBERTa-based Cross-Encoder, other pre-trained language models such as BERT and ELECTRA can be used as the backbone network of the reordering model.

[0064] Because of the use of multi-path recall, a large number of redundant or highly similar text blocks are inevitably generated (e.g., the same regulation appears in multiple documents). Directly feeding this duplicate content into a large model not only wastes valuable context window tokens, but may also interfere with the model's generation quality due to the high degree of information redundancy.

[0065] Therefore, in some preferred embodiments, this application further includes: deduplication of the relevant knowledge fragments, wherein the deduplication process includes similarity estimation based on minimum hash local sensitive hashing and redundancy recovery based on semantic clustering.

[0066] Specifically, the MinHash LSH algorithm (128 signature bands, Jaccard similarity threshold of 0.85) is used to quickly identify nearly identical text blocks; semantic clustering is used to classify highly similar text blocks, retaining the most representative text blocks in each cluster. For removed fragments, if they contain unique entities (not appearing in other fragments), the fragments are retrieved to prevent the loss of key information.

[0067] The retrieved text blocks are often fragmented, sometimes even truncated or lacking context due to segmentation. If answers are generated directly from these fragmented text blocks, large models may misunderstand or misinterpret the information due to a lack of contextual information.

[0068] To address this issue, in some preferred embodiments, this application implements a context aggregation mechanism, including: anchoring chapters based on the directory hierarchy path metadata, supplementing the content of the upper-level chapters along the directory hierarchy, and semantically compressing the aggregated context.

[0069] Specifically, the system extracts sibling headings and context paragraphs by using the directory hierarchy path in the text block metadata. If the current header / footer contains continuation words such as "continue from previous table" or "go to next page," tables spanning multiple pages are automatically merged. A summarization model is used to semantically compress the concatenated long text, preserving key information while controlling the length of the context.

[0070] S4: Generate an answer based on the relevant knowledge fragments using the multimodal large model, and bind the unique identifier of the relevant knowledge fragment and the directory hierarchy path metadata to the assertions in the answer. The enhanced user query and related knowledge fragments are input into the multimodal large model to generate a natural language answer.

[0071] During the generation process, the system binds the unique identifier of the relevant knowledge fragment and the directory hierarchy path metadata to the assertions in the answer, enabling verifiable citation tracing. Specifically, when generating the answer, the system records the unique identifier (such as a text block ID) of the relevant knowledge fragment on which the generation is based, and extracts the directory hierarchy path metadata associated with that knowledge fragment (such as "Chapter 2 > Section 2.1 > Paragraph 3").

[0072] For example, in response to a user query "Who is the contractor (Party B) in the construction contract for the Xiyuan Water Purification Plant?", the system generates the answer: "The contractor (Party B) in the construction contract for the Xiyuan Water Purification Plant is XX Construction Group. [Reference: Document ID: doc_xxx, Chapter: Contract Homepage, Block: CHK_001]". Users can directly locate the corresponding position in the original text by clicking the reference link to verify the information.

[0073] By binding a unique identifier of the source text block and directory-level path metadata to each assertion in the answer, precise tracing of the answer to the original text is achieved. Users can directly locate the corresponding position in the original text for verification, effectively reducing the risk of large model illusion and improving the verifiability and credibility of intelligent search results.

[0074] In summary, this application achieves high efficiency, high accuracy, and high reliability in intelligent archival processing through technologies such as a unified capability foundation of a multimodal large model, a serial collaborative processing pipeline, the construction of a semantic-structural dual-constraint knowledge base, and multi-path hybrid retrieval and traceable answer generation. The system supports local deployment within an enterprise intranet environment, ensuring that core archival data is not leaked and meeting the requirements of archival management and data security regulations.

[0075] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.

Claims

1. A method for intelligent archival processing based on a multimodal large model, characterized in that: Includes the following steps: The input multi-format archives are preprocessed to generate structured text. Based on the multimodal large model, the semantic features of the structured text are mapped to a hierarchical classification label system to generate classification labels. Based on the classification labels, a metadata extraction template is selected to extract key entities and generate metadata labels. The annotated structured text is split into several semantic text blocks using a semantic-granular coupling splitting strategy. Based on the directory structure of the structured text, directory hierarchy path metadata is added, and a dense semantic vector index and a sparse keyword vector index are constructed as a two-level index. Receive user queries and perform multi-path hybrid retrieval based on the dual-layer index to obtain relevant knowledge fragments; Based on the relevant knowledge fragments, the multimodal large model is used to generate answers, and the unique identifiers of the relevant knowledge fragments and the directory hierarchy path metadata are bound to the assertions in the answers.

2. The intelligent archival processing method based on a multimodal large model according to claim 1, characterized in that, Also includes: The business rules engine performs compliance checks on the category tags and metadata tags, and iterative optimization is carried out through incremental learning and training based on human feedback.

3. The intelligent archival processing method based on a multimodal large model according to claim 1, characterized in that, The multimodal large model is fine-tuned for domain adaptability by injecting industry classification standards and historical archive data.

4. The intelligent archival processing method based on a multimodal large model according to claim 1, characterized in that, The preprocessing includes: using an optical character recognition module to perform image denoising, image tilt correction, document layout analysis, and scene text recognition on the multi-format archives.

5. The intelligent archival processing method based on a multimodal large model according to claim 1, characterized in that, The semantic-granular coupling splitting strategy includes: calculating the semantic similarity of adjacent text segments, splitting when the similarity is lower than a preset threshold, forcibly splitting when the length of a text block exceeds a preset upper limit, and merging with adjacent blocks when the length is lower than a preset lower limit.

6. The intelligent archival processing method based on a multimodal large model according to claim 1, characterized in that, The dense semantic vector index is constructed using a hierarchical navigable small-world graph index, and the sparse keyword vector index is constructed using an inverted index.

7. The intelligent archival processing method based on a multimodal large model according to claim 1, characterized in that, The multi-path hybrid retrieval includes: performing retrieval in parallel at the semantic and structural layers, reordering the data through a cross-coding reorderer, and performing referential resolution, entity information completion, and query intent clarification on the user query.

8. The intelligent archival processing method based on a multimodal large model according to claim 1, characterized in that, Also includes: The relevant knowledge fragments are deduplicated, and the deduplication process includes similarity estimation based on minimum hash local sensitive hashing and redundancy recovery based on semantic clustering.

9. The intelligent archival processing method based on a multimodal large model according to claim 1, characterized in that, Also includes: The relevant knowledge fragments are aggregated in context. The context aggregation includes anchoring chapters based on the directory hierarchy path metadata, supplementing the content of the upper-level chapters along the directory hierarchy, and semantically compressing the aggregated context.

10. An intelligent archival processing system based on a multimodal large model, characterized in that: include: The preprocessing module is configured to preprocess multi-format archives to generate structured text; The classification and labeling module is configured to use a multimodal large model to classify the structured text and generate classification labels, and select a metadata extraction template based on the classification labels to generate metadata labels. The knowledge base construction module is configured to use a semantic-granular coupling splitting strategy to divide the annotated structured text into blocks and attach directory-level path metadata, and construct a dense semantic vector index and a sparse keyword vector index as a two-layer index. The question-and-answer retrieval module is configured to perform multi-path hybrid retrieval based on the dual-level index to obtain relevant knowledge fragments, generate answers using the multimodal large model, and bind the unique identifier of the relevant knowledge fragments and the directory-level path metadata.

Images