Intelligent knowledge extraction and structuring method for modern assembly type historical literature

Abstract

The invention provides an intelligent knowledge extraction and structuring method for modern assembly type historical literatures, relates to the technical field of artificial intelligence and natural language processing, and solves key problems in an existing text processing flow by constructing a closed-loop processing flow formed by layout analysis, primary generation, diagnosis and correction. The method comprises the following steps: firstly, recombining discrete OCR text lines into semantic blocks with a father-child nesting relationship through a text recombining technology based on hierarchical indexes; guiding the large model to generate preliminary structured data by utilizing a body self-adaptive cue word adapter; then, an automatic auditing module is introduced to carry out logic and evidence backtracking check on the data; and finally, a directional correction mechanism is triggered for audit abnormity, and high-precision knowledge extraction is realized. According to the method, the accuracy of the complex literature in the aspects of entity recognition, relationship construction and evidence tracing is remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence and natural language processing, in particular to a knowledge intelligent extraction and structuring method for modern assembly type historical literature. BACKGROUND

[0002] With the deepening of the digitalization and collation of ancient books, a large number of historical literature is assembled and published in a modern printing manner. Although this kind of assembly type literature is superior to traditional ancient books in printing quality and legibility of characters, its page structure is often extremely complex: a multi-level numbering system (such as "1.", "(1)" etc.) is generally used to distinguish the contents of different levels; the main text and the annotations (such as "editor's note") are frequently mixed, and often appear to be semantically broken across pages due to the reasons of layout; meanwhile, there are features such as mixed use of regular and simplified Chinese characters, and various expressions of historical proper nouns in the literature, which bring serious challenges to automatic knowledge extraction.

[0003] At present, the common method for literature digitization is to use optical character recognition (OCR) technology to convert scanned images into text. However, general OCR technology can usually only realize character-level recognition, and the output result is a sequence of discrete and logically unrelated text lines, which cannot understand the inherent hierarchical structure and semantic association within the literature. Directly inputting such raw text into a large language model for information extraction has significant defects: on the one hand, layout noise (such as page headers, page numbers) and the mixed arrangement of annotations and main text will seriously interfere with the model's judgment of content boundaries, leading to confusion in semantic understanding; on the other hand, large language models are prone to problems such as entity category misjudgment, relationship extraction errors, and even fabrication of fictitious citations when dealing with long texts and complex narratives, which seriously affects the accuracy and credibility of the generated structured knowledge.

[0004] In view of the above, the present application is proposed. SUMMARY

[0005] The present application provides a knowledge intelligent extraction and structuring method for modern assembly type historical literature, which can at least partially improve the above-mentioned problems.

[0006] To achieve the above-mentioned purpose, the present application adopts the following technical solutions:

[0007] A knowledge intelligent extraction and structuring method for modern assembly type historical literature, comprising: obtaining a scanned image of the literature to be processed, using an optical character recognition engine to extract the scanned image of the literature, obtaining raw text data, and performing layout analysis and processing on the raw text data to generate standardized text blocks; loading a corresponding prompt word adapter according to the genre characteristics of the literature, and calling a first large language model to analyze the standardized text blocks and generate preliminary structured data; The pre-built quality inspection module is invoked to perform logical verification and evidence backtracking checks on the preliminary structured data, and a diagnostic report is generated. When an anomaly is detected in the diagnostic report, correction instructions are constructed based on the diagnostic results, and the second language model is called to reconstruct and repair the anomaly, outputting the final structured knowledge data.

[0008] In summary, this invention provides a knowledge intelligent extraction method based on a "layout parsing and two-stage correction" architecture. This method effectively handles the complex layouts of modern compilation documents, suppresses the illusion of large models, and achieves high-precision structured knowledge production. It aims to solve the difficulties in semantic parsing caused by the complex page hierarchy and the mixing of annotations and main text in modern typesetting of ancient documents, as well as the problems of illusions, omissions, and errors easily generated by large language models in the extraction of long text information.

[0009] Specifically, firstly, in the digital normalization and structured reorganization step, the original text after OCR recognition undergoes layout analysis based on typesetting features. Noise such as headers and page numbers is identified and removed. Hierarchical numbering features are used as logical anchors to identify parent-child nesting relationships within the text, and annotation content is forcibly attached to the corresponding entries, thereby reorganizing discrete text lines into semantically complete standardized text blocks. Next, in the adaptive initial extraction step, a prompt word adapter with built-in predefined structured pattern constraints is loaded according to the document genre, guiding the first large-scale language model to parse the text blocks and generate preliminary structured data containing entities and relationships. Subsequently, in the automated diagnostic auditing step, a pre-built quality inspection module performs logical verification and evidence backtracking checks on the preliminary data, generating a diagnostic report containing anomalies. Finally, in the targeted correction step, correction instructions are constructed based on the diagnostic report. A second large-scale language model with stronger instruction compliance capabilities is invoked to locally rewrite and repair specific erroneous segments in the preliminary data, outputting the final high-precision structured knowledge data. This invention significantly improves the accuracy and reliability of complex documents in entity recognition, relationship construction, and evidence tracing by organically combining rule-based logical reorganization, domain constraint injection, and heterogeneous model collaborative error correction, providing an effective solution for efficient and reliable knowledge processing of compiled historical documents. Attached Figure Description

[0010] Figure 1 This is a flowchart illustrating the intelligent knowledge extraction and structuring method for modern compiled historical documents provided in this embodiment of the invention.

[0011] Figure 2 This is a schematic diagram of the framework for the intelligent knowledge extraction and structuring method for modern compiled historical documents provided in this embodiment of the invention.

[0012] Figure 3This is a schematic diagram comparing the effects of digital regularization and structured reorganization provided in the embodiments of the present invention.

[0013] Figure 4 This is a digital text engineering processing pipeline diagram provided in an embodiment of the present invention.

[0014] Figure 5 This is a schematic diagram of the architecture of the genre-adaptive prompt word adapter provided in an embodiment of the present invention.

[0015] Figure 6 This is a timing diagram of the two-stage diagnostic and correction extraction interaction provided in an embodiment of the present invention. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0017] refer to Figure 1 , Figure 2 As shown, the first embodiment of the present invention discloses a knowledge intelligent extraction and structuring method for modern compiled historical documents, which can be executed by a knowledge intelligent extraction and structuring device for modern compiled historical documents (hereinafter referred to as the extraction and structuring device), specifically, by one or more processors within the extraction and structuring device, to implement the following method: S1. Obtain the scanned image of the document to be processed, use an optical character recognition engine to extract the scanned image of the document to obtain the original text data, and perform layout analysis processing on the original text data to generate standardized text blocks. Specifically, step S1 further includes: acquiring the scanned image of the document to be processed, and extracting the original text data by using an optical character recognition engine; Based on the layout features, the original text data is analyzed and processed to identify and remove non-text elements, including headers and page numbers. Using hierarchical numbering features as logical anchors, the parent-child nesting relationship in the original text data after layout analysis is identified, and discrete text lines are reorganized into semantically complete normalized text blocks.

[0018] The layout analysis and processing specifically involves: establishing a hierarchical index regularity library, using the first feature to identify first-level entries, and using the second feature to identify second-level entries, wherein the first feature is the first-level hierarchical anchor point of the document, and the second feature is the sub-level logical boundary point; Establish annotation association rules to identify paragraphs that begin with configurable annotation identifiers and force them to be attached as attribute fields to the nearest preceding sub-entry, preventing annotation content from disrupting the semantic coherence of entries.

[0019] In this embodiment, unlike traditional image restoration, this step focuses on page layout analysis based on typesetting features after obtaining the basic text using OCR. By recognizing hierarchical numbers and "proofreading notes" identifiers, the flat text flow is reorganized into semantic blocks with parent-child nesting relationships, effectively solving the problem of annotations cutting off the main text. Specifically, a scanned image of the document to be processed is obtained. For example, a single-page scanned image of a modern compilation of historical documents is selected as input. This page contains entry text organized with multi-level numbering and a complex layout structure with mixed main text and annotations. Subsequently, an Optical Character Recognition (OCR) engine is used to extract text from the scanned image of the document, obtaining raw text data containing text content and its page position information. Next, in-depth typesetting analysis processing is performed on the raw text data. The core of this processing is to identify and remove non-text elements in the raw text data based on typesetting features, including area or column prompts in the header and page numbers. By filtering out these physical typesetting noises, a clean text foundation is provided for subsequent semantic analysis. Subsequently, the system performs a crucial logical reorganization operation: using the inherent hierarchical numbering features in the document as logical anchors to identify the implicit parent-child nesting relationships in the text. Specifically, a hierarchical index regular expression library is established, using the first feature (e.g., the first-level hierarchical anchor of the document (such as "1. a parent entry title...") to correspond to the parent entry or main entry entity. In the script, this is represented by the regular expression r"^(\d+[..].+)$".) to identify the first-level entry representing the parent entry entity or the overall document entry. Before detecting the next anchor, subsequent lines of text are continuously pushed into the semantic buffer of the current entry; at the same time, the second feature (child-level logical demarcation points (such as (1), (2))) is used to correspond to specific knowledge nodes (such as an event, person entry, or epigraph entry described by a certain child entry). In the script, this is represented by the regular expression r"^\s \s $"。〕Identifies secondary entries representing specific knowledge nodes. By identifying these two levels of anchors, seemingly flat text flows can be segmented and organized into semantic units with a clear hierarchical structure.

[0020] To further address the semantic breakage caused by the mixing of main text and footnotes (such as "Proofreading Notes") in compiled documents, the typesetting analysis and processing also established footnote association rules. Instead of direct extraction, these rules are first standardized using a "buffer mounting algorithm." When a paragraph beginning with a configurable footnote identifier such as "Proofreading Notes:" is identified, attribute placement logic is triggered. This logic forces the footnote content to be treated as an attribute field and mounted under the most recently identified preceding sub-entry (such as the most recent "(N)" numbered entry), rather than allowing it to become a separate paragraph or be mistakenly mounted under other entries. For example, a biographical footnote about a person is accurately associated with its corresponding sub-entry object. This mechanism fundamentally solves the interference of physical typesetting noise on semantic understanding, effectively preventing footnote content from disrupting the original semantic flow between entries and ensuring the accuracy of knowledge attribution.

[0021] Ultimately, through a series of operations including typesetting feature-based recognition, noise removal, hierarchical anchor point positioning, and forced attachment of annotations, the system successfully reconstructed the discrete, unstructured text line sequence output by the OCR into semantically complete and structurally clear standardized text blocks. Each text block clearly defines the first-level entry, its subordinate second-level entries, and the accompanying text and annotations, providing high-quality and deterministic input for subsequent knowledge extraction steps. This step fundamentally improves the system's parsing accuracy for documents with complex layouts and is a crucial foundation for the reliable execution of subsequent intelligent processing.

[0022] In simple terms, after acquiring the raw text data, the planar text stream is divided into (K) semantically closed data blocks using a "hierarchical index regular expression library." This enables the adapter loaded in step S2 to accurately locate knowledge boundaries, achieving a leap from "messy text" to "computable semantic units." The specific layout analysis logic based on typographic features is shown in Table 1.

[0023] Table 1. Logic Comparison Table for Page Layout Analysis Based on Typography Features

[0024] S2, based on the genre characteristics of the document, load the corresponding prompt word adapter, and call the first major language model to parse the normalized text block to generate preliminary structured data; Specifically, step S2 further includes: loading a corresponding prompt word adapter according to the preset genre characteristics of the document, wherein the prompt word adapter has a predefined structured pattern built in as a constraint condition, and the genre characteristics are obtained through deep modeling of the narrative norms of historical documents; Based on the prompt word adapter, the first major language model is invoked to parse the normalized text block, generating preliminary structured data containing entities and relationships.

[0025] The predefined structured pattern includes a dual-track naming strategy and a relation constraint strategy. The dual-track naming strategy includes defining a standardized name field to store normalized entity names and a source name field to store the original document description. The relation constraint strategy refers to explicitly defining the set of controlled relations between entities in the prompt words and requiring the model to distinguish the types of behavioral relations under different institutional contexts in order to improve the consistency and transferability of relation extraction.

[0026] The first major language model serves as the foundation for semantic understanding, and it can employ the GLM model (GLM-4.5). Its advantage lies in its deep semantic understanding capabilities for Chinese and classical Chinese corpora. In step S2, it is responsible for handling complex historical document narratives, accurately parsing implicit logical relationships such as administrative behavior relationships and document citation relationships. Although it occasionally fluctuates in adhering to the extremely complex JSON format, it can guarantee the accuracy of content understanding.

[0027] In this embodiment, an adaptive initial extraction step is implemented. A corresponding prompt adapter is loaded based on the genre characteristics of the document, and a built-in structured pattern serves as a constraint, guiding the first large language model to generate preliminary data. Specifically, firstly, a corresponding prompt adapter is loaded based on the genre characteristics of the target document; where the adapter refers to a structured instruction template (PromptAdapter) oriented towards a specific document genre. The genre characteristics originate from deep modeling of the narrative norms of specific types of historical documents (such as compilations of epigraphs, inscriptions, or archival documents), enabling the system to invoke the most suitable processing logic for different document types. The core of the prompt adapter lies in its built-in predefined structured patterns. These patterns, acting as strong constraints (Schema constraints), aim to precisely guide the originally divergent generation capabilities of the general large language model onto a slot-filling track that conforms to domain norms, thus serving as a logical mold for knowledge production.

[0028] Specifically, the predefined structured patterns mainly include the following key strategies: Dual-track naming strategy: This strategy aims to resolve the contradiction between entity alignment and the preservation of historical authenticity in knowledge graph construction. The adapter explicitly requires that for different representations of the same entity in documents (e.g., the real name and courtesy name of a historical figure, different names of places in ancient and modern times, full name and abbreviation of an organization), two fields must be written: First, the normalized standard name is uniformly written in the `canonical_name` field for machine retrieval and graph connection to ensure node uniqueness and connectability; second, the original string recognized by OCR is retained in the `name_trad` field for subsequent tracing and verification. This strategy ensures data standardization while fully preserving the historical features of the historical materials.

[0029] Relational Constraint Strategy: Addressing the complexity of social relationships and behavioral types in historical documents, the adapter explicitly defines a controlled set of relationships in the prompts and forces the model to make judgments and extractions within this set. For example, it clearly distinguishes between actions such as announcements and official reports (belonging to the context of administrative documents) and actions such as donations and fundraising for repairs (belonging to the context of public participation). This explicit constraint effectively prevents the model from confusing or misjudging relationship types in complex historical contexts, improving the academic accuracy of relationship extraction.

[0030] In addition, entity type constraints can be included, that is, the prompt word stipulates that "if the original text contains multiple names or categories of the same type of sub-entry object, they should still be uniformly written into the predefined standard type field value", thus realizing the standardization of data.

[0031] Subsequently, based on the complete instructions constructed by the loaded prompt word adapter, the first major language model is invoked to parse and generate normalized text blocks. In this embodiment, a model with extensive pre-training experience on Chinese and classical Chinese corpora (such as the GLM model) is selected as the semantic understanding foundation. Its advantage lies in its strong ability to understand deep semantics and implicit logical relationships in historical documents (such as the aforementioned distinction between official and unofficial behavior). Under the strict pattern constraints of the adapter, the model processes each normalized text block and outputs preliminary structured data (usually presented in structured formats such as JSON) that conforms to a predetermined pattern and contains entities, attributes, and their relationships.

[0032] Based on this step, this invention achieves a leap from regularized text to preliminary structured knowledge. The genre-adaptive prompt adapter mechanism greatly enhances the flexibility and scalability of the method, making it applicable to various types of documents. The injection of a dual-track naming system and relational constraint strategy standardizes the output format and content logic of the data from the source, laying the foundation for generating high-quality, standardized data that can be directly used for knowledge graph construction. It also significantly reduces the potential for inconsistencies in representation and chaotic relationships that may arise in large models during open generation.

[0033] S3, calls the preset quality inspection module to perform logical verification and evidence backtracking check on the preliminary structured data, and generates a diagnostic report; Specifically, step S3 further includes: comparing the number of sub-entries and the number of hierarchical numbers in the preliminary structured data, determining whether the two are consistent, and generating an entry number consistency check result; Check whether the content of the source field in the preliminary structured data is a real text fragment from the literature, exclude fictitious citations generated by the large model, and generate citation evidence verification results; A diagnostic report is generated based on the consistency of the number of entries and the verification of citation evidence.

[0034] In this embodiment, the diagnostic auditing steps are automated. A pre-built quality inspection module performs logical verification (such as item counting) and evidence backtracking checks on the preliminary data, generating a diagnostic report. Specifically, firstly, an item count consistency check is performed. The number of sub-items in the preliminary structured data (such as a JSON array) is precisely compared with the number of original document sub-item numbers identified and counted based on a hierarchical index regularity library (denoted as the K value). This check constructs a logical closed loop throughout preprocessing and postprocessing. If the two counts are inconsistent, it directly indicates that the large language model may have omissions or redundancies during the generation process, which is judged as a "structural illusion" or error, and a corresponding anomaly is generated in the diagnostic report. This mechanism effectively solves the common omission problem in long text information extraction, ensuring that the output results strictly correspond to the original text in macroscopic structure.

[0035] In simple terms, the quantity consistency constraint means that the prompt explicitly requires "if there are K events in total, there must be exactly K events...event IDs and entry numbers must correspond one-to-one". This uses the number of entries (K) calculated in step S1 as prior knowledge, forcing the LLM to output an equal number of event objects, effectively solving the problem of missing long texts. This is the key logical loop connecting steps S1 and S3. The counter output value K in step S1 (e.g., if 6 (N) labels are identified) serves as the "expected truth value" in step S3. If the length of the JSON array is ≠ K, the system directly determines it as a "structural illusion" and triggers a correction.

[0036] Secondly, citation evidence verification is performed. To address the issue of potential fabrication or misinterpretation of source information by large language models, the authenticity of source-annotated evidence fields in the initial structured data is verified. The verification logic is as follows: the text content in this field is treated as a string and subjected to a full-text match search within the standardized text block output from step S1. If no match is found or the similarity is below a preset threshold, the citation is determined to be fictitious content generated by the model and marked in the diagnostic report. This "original text backtracking matching" mechanism fundamentally eliminates the risk of citation inaccuracies in knowledge production, ensuring the traceability and academic credibility of structured knowledge.

[0037] In short, to address the issue of easily fabricated citations in large models, a "backtracking matching" process is performed. This checks whether the content of the `description_source` field in the JSON can be found in the normalized text block output in step S1 using string matching. If the match fails (Distance > threshold), it is marked as a fabricated citation.

[0038] Finally, by synthesizing the results of the item quantity consistency check, the verification results of citation evidence, and other built-in rule checks (such as the preliminary screening of entity types based on domain knowledge), a structured diagnostic report is automatically generated. This report clearly lists all the detected anomalies, such as "Missing items: 6 were expected, but only 5 were generated", "Fictitious citation: The content in field X was not found in the original text", etc., and may be accompanied by error type and location information.

[0039] Briefly, in actual use, a diagnostic report is generated using the DIAGNOSTIC_PROMPT. For example, when checking the "Trade name / Institution naming characteristics", if it is found that entities ending with "hao, xing, ji" are misjudged as individuals, they are marked as errors. This step is not a simple manual verification, but a hybrid mode based on "rule engine + prompt auditing". Instead, the preliminary JSON and the original text are input into the large language model, and the DIAGNOSTIC_PROMPT (diagnostic prompt) is loaded. Among them, the recognition logic of anomalies is as follows: a "reverse checklist" is built into the prompt. For example, when checking the "Trade name naming characteristics" (such as ending with "xing, ji"), if it is found that such entities are classified as Person, a CRITICAL_ISSUE will be generated in the diagnostic report.

[0040] S4. When it is determined that there are anomalies in the diagnostic report, a correction instruction is constructed based on the diagnostic results, and the second large language model is called to reconstruct and repair the anomalies, and the final structured knowledge data is output.

[0041] Specifically, step S4 further includes: when it is determined that there are anomalies in the diagnostic report, the normalized text block, the preliminary structured data, and the diagnostic report are combined to generate a correction context, and the correction context is input into the second large language model for reconstructing and repairing the anomalies; Among them, the reconstruction and repair process includes entity classification correction, citation discrimination correction, and relationship optimization.

[0042] The second large language model is the correction base. Among them, the second large language model is configured to execute the directional reconstruction and repair instructions based on the diagnostic report. The second large language model can adopt the Doubao model (Doubao-Seed-1.6). This model has extremely strong instruction-following ability and is extremely sensitive to the structural constraints of JSONSchema. In the stage of step S4, it does not need to re-understand the profound historical background of the full text, but strictly follows the correction instructions in the CORRECTION_PROMPT to fix format errors or entity classification errors at specific points.

[0043] In this embodiment, a targeted correction step is performed. For anomalies found in the diagnostic report, the system invokes instructions to perform targeted repair using a more powerful second language model. In practical use, `CORRECTION_PROMPT` can be used, inputting the combination of [original text + erroneous JSON + diagnostic report] into the second model. Specifically, when the system determines that there are anomalies in the diagnostic report that need correction, it does not re-process globally, but instead initiates an efficient, local rewriting mechanism based on the diagnostic results to output the final high-quality structured knowledge data. First, precise correction instructions are constructed based on the diagnostic results. Then, three key pieces of information are combined to generate a correction context: normalized text blocks, preliminary structured data, and the diagnostic report. This correction context, along with the specific correction requirements, is then input into the second language model for repair.

[0044] Specifically, entity classification correction involves the second model moving misclassified entities to the correct array based on the prompt "business names...should be classified as institutions". Citation identification correction involves correcting incorrect entity types based on the rule "book titles...strictly prohibited from being identified as locations or institutions". Relationship optimization involves correcting the relationship direction based on the instruction "geographical hierarchy...direction must be child → parent". The adapter constraint and dual-model collaboration mechanism are shown in Table 2.

[0045] Table 2 Adapter Constraints and Bimodel Collaboration Mechanism

[0046] To further illustrate the beneficial effects of this invention, let's take the processing of epigraph entries in a modern compilation of historical documents as an example. These documents typically employ modern typographical formats, containing multi-level headings and complex annotations, and can represent the layout and narrative complexity of various authentic historical documents.

[0047] Example 1: Digital regularization and structured reorganization (corresponding to...) Figure 3 For document pages (such as pages containing parent entries and their child entries), which include multi-level headings and complex footnotes, this step processes them as follows: Page layout noise removal: The system first calls the OCR engine to obtain the original text and coordinate information. Based on geometric position rules, it automatically identifies and removes regional header text at the top of the page and page numbers at the bottom, retaining only the main text area.

[0048] Hierarchical logical reorganization: (1) First-level index recognition: The algorithm uses regular expressions to scan the beginning of the line, identifies "1. Title of a parent entry (quantity marker)" as the parent entity, and extracts the quantity information in the parentheses as the verification benchmark. (2) Second-level index recognition: The algorithm uses the feature of parentheses and numbers to identify "(1)", "(2)", etc. as child entities. (3) Note mounting: The algorithm identifies note paragraphs that start with "Note:", "Proofreading notes:", "Note:", etc. The system forces the text to be assigned to the attribute field of the nearest child entry based on the layout indentation and context logic, avoiding misjudgment as main text content.

[0049] Output standardized text: After the above processing, the original OCR text stream is transformed into text blocks with nested structures, clearly distinguishing the title, sub-entries, author, main text content, and footnotes.

[0050] Example 2: Adaptive initial extraction addresses the complexities of epigraph-type entries or short entry references by loading a dedicated adapter for short entry formats. The adapter defines strict pattern constraints, requiring that the year in the comment section be extracted as the creation time, and that biographical information be extracted as a biographical summary.

[0051] Example 3: Two-stage diagnosis and correction Initial generation: The first model (such as GLM-4) is called to process the above text block. The model correctly extracts the sub-entry content and author field information of entry (1).

[0052] Automated auditing: When processing subsequent entries, the auditing module found that the footnote of entry (2) contained "according to a certain reference, 'a certain book title'...". In the initially generated structured data, the model incorrectly extracted the noun in the cited phrase as an "institution" entity and established an inappropriate association. At the same time, the system found through entry counting that the total number of generated entries was less than the total number of original text numbers, which was suspected to be an omission.

[0053] Diagnostic Report: The audit rule base detected that book titles usually represent sources of literature, and "According to..." is a typical citation structure. The system generated a diagnostic report indicating that the entity type of "Example Institution Name A" may be incorrect, and that there are missing entries.

[0054] Targeted Correction: The system activates the correction command, inputting the original text fragment, erroneous structured data, and diagnostic report into the second model. The second model corrects the relevant entity type errors according to the command, moves them to the evidence source attribute, and completes the missing sub-entry (3) "Example Title C", so that the final output is consistent with the original text hierarchical numbering logic.

[0055] In summary, compared with existing technologies, this method has the following advantages: 1. Improved parsing accuracy for complex layouts. Through regularized hierarchical reorganization, it successfully solves the semantic breakage problem caused by mixed annotations and main text, ensuring the accuracy of knowledge attribution. 2. Guaranteed data credibility and compliance. Through a closed-loop mechanism of "initial generation—auditing—correction," and utilizing the collaborative work of two models, it significantly reduces the illusion rate of the large model, ensuring the authenticity and traceability of citations, and that the output format strictly conforms to predefined standards. 3. Achieved efficient processing of multi-source heterogeneous data. The genre-adaptive adapter mechanism allows this method to be flexibly applied to the processing of various types of documents such as archives, epigraphs, and inscriptions.

[0056] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for intelligent knowledge extraction and structuring of modern compiled historical documents, characterized in that, include: The process involves acquiring scanned images of the documents to be processed, extracting the raw text data using an optical character recognition engine, and then performing layout analysis on the raw text data to generate standardized text blocks. Specifically: The scanned image of the document to be processed is acquired, and the original text data is extracted from the scanned image using an optical character recognition engine. Based on the layout features, the original text data is analyzed and processed to identify and remove non-text elements, including headers and page numbers. Using hierarchical numbering features as logical anchors, we can identify the parent-child nesting relationship in the original text data after page layout analysis and reorganize discrete text lines into semantically complete normalized text blocks. Based on the genre characteristics of the document, the corresponding prompt word adapter is loaded, and the first major language model is called to parse the normalized text block to generate preliminary structured data; The pre-built quality inspection module is invoked to perform logical verification and evidence backtracking checks on the preliminary structured data, and a diagnostic report is generated. When an anomaly is detected in the diagnostic report, correction instructions are constructed based on the diagnostic results, and the second language model is called to reconstruct and repair the anomaly, outputting the final structured knowledge data.

2. The knowledge intelligent extraction and structuring method for modern compiled historical documents as described in claim 1, characterized in that, The specific page layout analysis and processing is as follows: Establish a hierarchical index regular expression library, use the first feature to identify first-level entries, and use the second feature to identify second-level entries. The first feature is the first-level hierarchical anchor point of the document, and the second feature is the sub-level logical boundary point. Establish annotation association rules to identify paragraphs that begin with configurable annotation identifiers and force them to be attached as attribute fields to the nearest preceding child entry.

3. The knowledge intelligent extraction and structuring method for modern compiled historical documents according to claim 1, characterized in that, Based on the genre characteristics of the document, the corresponding prompt word adapter is loaded, and the first major language model is called to parse the normalized text block to generate preliminary structured data, specifically: Based on the preset genre characteristics of the documents, the corresponding prompt word adapter is loaded. The prompt word adapter has a predefined structured pattern built in as a constraint condition. The genre characteristics are obtained through deep modeling of the narrative norms of historical documents. Based on the prompt word adapter, the first major language model is invoked to parse the normalized text block, generating preliminary structured data containing entities and relationships.

4. The knowledge intelligent extraction and structuring method for modern compiled historical documents according to claim 3, characterized in that, The predefined structured pattern includes a dual-track naming strategy and a relation constraint strategy. The dual-track naming strategy includes defining a standardized name field to store normalized entity names and a source name field to store the original document description. The relation constraint strategy refers to explicitly defining a set of controlled relations between entities in the prompt words, requiring the model to distinguish the types of behavioral relations under different institutional contexts, so as to improve the consistency and transferability of relation extraction.

5. The knowledge intelligent extraction and structuring method for modern compiled historical documents according to claim 1, characterized in that, The pre-built quality inspection module is invoked to perform logical verification and evidence backtracking checks on the preliminary structured data, generating a diagnostic report, specifically: The number of sub-entries in the preliminary structured data is compared with the number of hierarchical numbers to determine whether they are consistent, and an entry number consistency check result is generated. Check whether the content of the source field in the preliminary structured data is a real text fragment from the literature, exclude fictitious citations generated by the large model, and generate citation evidence verification results; A diagnostic report is generated based on the consistency of the number of entries and the verification of citation evidence.

6. The method for intelligent knowledge extraction and structuring of modern compiled historical documents according to claim 1, characterized in that, When an anomalies are detected in the diagnostic report, correction instructions are constructed based on the diagnostic results. The second major language model is then invoked to reconstruct and repair the anomalies, outputting the final structured knowledge data, specifically: When an anomaly is detected in the diagnostic report, the normalized text block, preliminary structured data and diagnostic report are combined to generate a correction context, and the correction context is input into the second language model to reconstruct and repair the anomaly. The reconstruction and repair process includes entity classification correction, citation identification correction, and relationship optimization.

7. The knowledge intelligent extraction and structuring method for modern compiled historical documents according to claim 1, characterized in that, The first large language model serves as the semantic understanding base, while the second large language model serves as the correction base. The instruction-following capability of the second large language model is superior to that of the first large language model.

8. The method for intelligent knowledge extraction and structuring of modern compiled historical documents according to claim 1, characterized in that, The first largest language model uses the GLM model, and the second largest language model uses the Doubao model.

Images