Electronic archive information extraction method and extraction system

Abstract

The invention relates to the technical field of information extraction, in particular to an information extraction method and system for electronic archives. The method comprises the following steps: obtaining a to-be-processed electronic file and carrying out OCR identification to generate initial text data; error detection is carried out on the initial text data, and OCR error recognition candidate items in the initial text data are recognized; for each OCR misrecognition candidate item, generating a first data name according to context semantics and a layout structure of the candidate item; extracting low-level features of the first data name, and performing named entity recognition on each OCR misrecognition candidate item by utilizing a preset field word list in combination with the first data name; through a four-in-one process of ''misrecognition detection + named entity recognition + semantic error correction + templated extraction'', the core technology bottlenecks of inaccurate recognition, poor error correction capability, low information extraction intelligence and the like in the prior art are solved, and the accuracy, stability and intelligent level of electronic archive information extraction are remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of information extraction technology, and in particular to a method and system for extracting information from electronic archives. Background Technology

[0002] Early electronic document information extraction relied primarily on manual input and simple text matching techniques, which were inefficient and prone to errors. Subsequently, the introduction of Optical Character Recognition (OCR) technology made automatic image text recognition possible, significantly improving the efficiency of electronic document structuring. However, traditional OCR algorithms suffer from low accuracy when dealing with complex layouts, handwritten text, and low-resolution images, leading to frequent misidentifications and affecting the reliability of subsequent information extraction. In recent years, with the development of Natural Language Processing (NLP) and deep learning technologies, semantic understanding and context modeling capabilities have significantly improved, making semantic analysis-based information extraction a research hotspot. Furthermore, multimodal fusion technologies such as layout structure analysis, entity recognition, and naming rule reasoning have emerged, effectively enhancing the automatic identification and extraction of key information from complex documents. However, traditional OCR technology is highly susceptible to misidentification when dealing with complex layouts, overlapping characters, noise interference, handwritten text, or poor image quality, resulting in incomplete or erroneous text data, affecting the quality of information extraction. Moreover, many existing systems cannot correct errors based on context, often failing to correct homophones or semantically similar words. Summary of the Invention

[0003] Therefore, it is necessary to provide a method and system for extracting information from electronic records in order to solve at least one of the above-mentioned technical problems.

[0004] To achieve the above objectives, a method for extracting information from electronic archives is provided, the method comprising the following steps:

[0005] Step S1: Obtain the electronic document to be processed and perform OCR recognition to generate initial text data; perform error detection on the initial text data to identify OCR misrecognition candidates;

[0006] Step S2: For each OCR misidentification candidate, generate a first data name based on its contextual semantics and layout structure; extract the low-level features of the first data name, and use a preset domain vocabulary in combination with the first data name to perform named entity recognition on each OCR misidentification candidate, thereby assigning a second data name.

[0007] Step S3: Utilize the dual mapping relationship between the first data naming and the second data naming to automatically correct and re-identify the OCR misidentified candidates, generating corrected and accurate text data;

[0008] Step S4: Extract key information from the corrected and accurate text data according to the preset information field template, and output structured information.

[0009] This invention effectively corrects the recognition errors of conventional OCR systems for similar characters, ambiguous text, and industry-specific terms by performing contextual semantic parsing and layout structure analysis on misidentified candidates in the initial OCR text, combined with a first data naming generation rule and a second data naming mapping strategy. This significantly improves the text recognition accuracy of electronic documents. Utilizing a dual mechanism of first data naming (based on context and layout structure) and second data naming (combined with semantic naming based on a domain thesaurus), the system achieves high-precision recognition and semantic reconstruction of key information entities such as names of people, places, organizations, times, and values, enhancing the semantic integrity of the text. The system not only detects significant recognition errors but also performs deep error correction analysis on words with similar structures but inconsistent meanings, possessing a certain degree of semantic fault tolerance. It is suitable for error correction scenarios in complex contexts, such as high-density professional texts like historical archives, official documents, and medical reports. By introducing configurable preset domain thesaurus and template libraries, the method can quickly adapt to the electronic document processing needs of different industries (such as legal, medical, engineering, and financial sectors), exhibiting good scalability and industry universality. By introducing a structured template extraction mechanism in the OCR post-processing stage, the identified and corrected text can be extracted according to field logic to output structured data, providing standardized data support for subsequent archiving management, retrieval analysis, and knowledge graph construction. This method automates the entire process of identification, error correction, naming, and structure extraction, eliminating the need for manual verification and modification, significantly reducing labor costs and processing time, and is particularly suitable for large-scale automated document processing tasks. Through semantic-driven OCR error correction and field reconstruction, the generated text is closer to the original semantics and format, enhancing the clarity, standardization, and retrieval value of document data in subsequent use. Therefore, this invention, through a four-in-one process of "misidentification detection + named entity recognition + semantic error correction + template extraction," solves the core technical bottlenecks of existing OCR technologies, such as inaccurate identification, poor error correction capabilities, and low information extraction intelligence, significantly improving the accuracy, stability, and intelligence level of electronic archive information extraction.

[0010] Preferably, step S1 includes the following steps:

[0011] Step S11: Obtain the electronic file to be processed;

[0012] Step S12: Perform image analysis and region division on the electronic archives to be processed, extract structured text image regions, and obtain image partition data;

[0013] Step S13: Perform OCR recognition on the image partition data to extract the text content in the electronic document and generate initial text data; perform character confidence analysis on the initial text data to extract the positions of characters with recognition confidence below a preset threshold and generate low-confidence character annotation data;

[0014] Step S14: Combine the initial text data with the low-confidence character annotation data, perform semantic integrity analysis, identify segments with semantic missing or logical interruptions, and generate semantic anomaly candidate segment data; perform intersection location on the low-confidence character annotation data and the semantic anomaly candidate segment data, annotate the OCR misidentification area, and generate OCR misidentification candidate data.

[0015] This invention employs a multi-layered processing flow combining image analysis, region segmentation, character confidence analysis, and semantic integrity analysis to achieve step-by-step misidentification detection from the image layer, character layer, and semantic layer. This significantly improves the accuracy of OCR misidentification candidate detection and avoids missed detections and false alarms. By segmenting electronic documents to extract structured text image regions, subsequent OCR recognition gains layout structure awareness, enhancing the consistency between text recognition and semantic judgment. This is particularly suitable for documents with complex layout structures, such as tables and official documents. By extracting characters with recognition confidence below a threshold and forming low-confidence character annotation data, the system can automatically identify high-risk areas for potential recognition errors, enabling targeted error correction strategies and improving processing efficiency and accuracy. This method, through semantic integrity analysis to identify logical interruptions and semantic omissions, can not only detect character-level errors but also identify semantic anomalies at the text fragment level, achieving a higher level of language error detection capability. By analyzing the intersection of low-confidence characters and semantically abnormal candidate segments, the system accurately identifies OCR misidentification areas. Compared to traditional methods based on a single confidence threshold, this significantly reduces the false positive rate and improves the efficiency and success rate of subsequent error correction algorithms. The entire S1 step system not only automatically completes the structured recognition process from image to text but also possesses intelligent semantic analysis and misidentification localization capabilities, greatly reducing the burden of manual review and providing strong foundational support for the automated processing of large-scale electronic documents.

[0016] Preferably, in step S13, OCR recognition is performed on the image partition data to extract the text content from the electronic document, including:

[0017] The image partitioning data is divided into image partitioning data blocks with a unit size of 128×128 pixels. Edge detection and connected component analysis are performed on each partition, and connected components with an area of ​​not less than 200 pixels are selected. 2 Suspected text areas with aspect ratios between 2:1 and 10:1;

[0018] Perform character-level OCR recognition in each suspected area. It supports recognizing characters with sizes ranging from 12pt to 36pt. For characters smaller than 10pt, double pixel interpolation is performed before recognition to obtain the recognized character information.

[0019] The identified character information is evaluated for confidence and marked as anomalies. Characters with a confidence level of less than 85% are marked as low-confidence characters. When the average confidence level of the entire text is higher than 92% and the recognition error rate is lower than 3%, it is marked as a valid text segment.

[0020] Based on the valid text segments, the original coordinates of the electronic file to be processed are mapped and the position is reconstructed. The images are then stitched and sorted according to the order from top to bottom and from left to right, and the output is structured initial text data with field numbers, character content, position information, character spacing and average confidence.

[0021] This invention divides image partition data into 128×128 pixel basic units, performs edge detection and connected component analysis within each unit, and combines an area threshold (≥200px) 2 The method filters suspected text areas using aspect ratio ranges (2:1 to 10:1), effectively filtering background noise and non-text areas, significantly improving the input quality and processing efficiency of the OCR recognition module. Character-level OCR recognition is performed within suspected text areas. Specifically, for characters smaller than 10pt, image quality is enhanced by doubling the pixel count before recognition, solving the problem of low accuracy in traditional OCR for small fonts and low-resolution characters. This enhances the robustness of the method in scenarios with dense small text, such as old documents, invoices, and contracts. A confidence level assessment is performed on each recognized character, and single characters with a recognition rate below 85% are marked as "low-confidence characters." Then, an average confidence level greater than 92% and an error rate less than 3% are set as the standard for "valid text segments," achieving a dual-layer confidence assessment system at the character and segment levels, ensuring the overall credibility of the output text from the source. The recognition results not only output character content but also combine valid text segments to perform original image coordinate mapping and position reconstruction. This ensures that each text segment possesses spatial positioning information from the original image, facilitating subsequent operations such as field alignment, content location, and visual review, thus improving the operability and traceability of structured text data. The output results are concatenated and sorted in a "top-to-bottom, left-to-right" order, conforming to human reading habits and effectively avoiding character order issues. This is particularly suitable for multi-column layouts or tabular documents, ensuring the semantic coherence and logical integrity of information extraction. The initial structured text data includes not only field numbers and character content but also extended attributes such as character spacing and average confidence score, providing more dimensions of data support for subsequent information extraction, field matching, and semantic analysis, enhancing the overall system's intelligent processing capabilities.

[0022] Preferably, in step S2, for each OCR misidentification candidate, generating a first data name based on its contextual semantics and layout structure includes:

[0023] For each OCR misidentified candidate, extract the corresponding preceding and following text lines to generate context text segment data;

[0024] Perform part-of-speech tagging and syntactic structure analysis on the context text segment data, extract sentence structure and keyword sequences, and generate semantic analysis result data;

[0025] By matching the OCR misidentification candidate data with the semantic analysis results data, and matching the field names or common phrases in the standard terminology database, semantic matching candidate field data is generated.

[0026] Extract the relative position, alignment mode, and font style of the OCR misidentified candidate data in the image partition data to generate page layout feature data;

[0027] The system integrates semantic matching candidate field data with layout structure feature data, performs context consistency scoring, and generates the first data name based on the highest score result.

[0028] This invention extracts the text lines preceding and following the location of misidentified OCR candidate options to generate contextual text segments. Further, it performs part-of-speech tagging and syntactic structure analysis to extract semantic keyword sequences and sentence structures. This allows for a broader understanding of the contextual meaning of each candidate option, significantly improving the consistency between the naming results and the original text. By combining a standard terminology database, semantic matching and reasoning are performed on the field names or common phrases corresponding to the OCR candidate options in the context, outputting semantically matched candidate field data. This enables automatic recognition of professional terms, domain-specific nouns, and fields with specific formats, effectively solving the problem of terminology misjudgment in OCR systems within specific industry documents. Furthermore, this invention not only relies on semantic information but also extracts the position coordinates, alignment relationships, font styles, and other layout structural features of the misidentified candidate options within image partitions, providing auxiliary information at the visual space level. This compensates for the ambiguity or unclear context in semantic judgment, enhancing the overall reliability of the naming generation. By integrating semantic matching results with layout structure feature data, and performing contextual consistency scoring, the naming inference not only conforms to the semantic background but also to the image structure features, realizing a multi-dimensional, consistency-driven accurate naming generation logic, effectively avoiding the recognition error of "semantically correct but misaligned layout".

[0029] Preferably, step S2 involves extracting low-level features from the first data naming and using a preset domain vocabulary in conjunction with the first data naming to perform named entity recognition on each OCR misidentification candidate, including:

[0030] The first data name is subjected to character-level encoding and sub-word segmentation to extract its low-level language features such as character length, stem structure and naming format, and generate naming low-level feature data;

[0031] The system uses named low-level feature data to match pre-defined domain terms and generates candidate domain entity label data based on rules.

[0032] The first data name and candidate domain entity label data are used to perform named entity classification scoring to identify misclassified or ambiguous entity ranges and generate entity confidence score data.

[0033] Based on the entity confidence score data, the final entity type is determined for each OCR misidentification candidate, and named entity label binding is performed to generate a second data name.

[0034] This invention extracts low-level linguistic features such as character length, stemming structure, and naming format from the naming of first data through character-level encoding and sub-word segmentation. This enables the system to recognize complex naming forms, including abbreviations, compound words, and words combining numbers and letters, thereby improving its adaptability to industry terms, proper nouns, and semi-structured fields from the source. Using a pre-defined domain lexicon, semantic word matching is performed on the low-level feature data, and candidate domain entity labels are generated based on rules. This effectively covers specific entity categories commonly found in the industry, such as names of people, places, numbers, and professional terms, strengthening the ability to recognize fields with ambiguous semantic boundaries or spelling variations, and enhancing the model's domain specialization. The system not only matches entities but also calculates named entity classification scores based on the semantic consistency and structural rationality between context and candidate labels. Entity confidence scores identify complex situations such as misclassification, name duplication, and field overlap, thereby avoiding the incorrect classification of irrelevant or ambiguous fields into the wrong category and effectively improving overall recognition accuracy. After scoring, each OCR misidentification candidate is bound to a final entity type label to form a second data name. The labeling information has the characteristics of strong stability, traceability and interpretability, providing accurate naming semantic anchors for subsequent error correction logic, field extraction and knowledge graph construction.

[0035] Preferably, the method for confirming the dual mapping relationship between the first data naming and the second data naming in step S3 includes:

[0036] The first and second data names are compared by position index, and the relative position mapping information of the corresponding items in the original text is extracted to generate named position mapping data.

[0037] Based on the naming position mapping data, a similarity measurement is performed on the character content of the first data name and the second data name to generate naming semantic similarity data;

[0038] Based on the naming semantic similarity data, name pairs with similarity higher than a preset threshold are selected, an initial bidirectional naming mapping table is constructed, and a candidate dataset for naming mapping is generated.

[0039] Conflict checking and ambiguity alignment are performed on the candidate datasets for naming mapping, and ambiguous naming bindings are filtered out, thereby obtaining a dual mapping relationship between the first and second data names.

[0040] This invention compares the original text position indexes of the first and second data names, extracts their spatial or logical relative positions in the context structure, and generates named position mapping data. This provides a clear positioning reference for subsequent semantic matching, solving the problem of mismatches that can easily occur when relying solely on semantic similarity. By calculating character-level and word-level similarity between named pairs, it supports handling complex text variations such as case variations, synonyms, and abbreviation replacements, improving the system's intelligent association and correspondence capabilities for non-standard fields and incomplete matches, and enhancing the robustness of the mapping relationship. Based on a similarity threshold, high-confidence named pairs are selected to construct an initial bidirectional naming mapping table, which supports not only forward lookups of "first name → second name" but also reverse checks of "second name → first name," improving the efficiency of subsequent naming correction, tag confirmation, and information consistency verification. For binding items where multiple named items point to the same target or have semantic overlap, conflict analysis and consistency judgment are used to eliminate ambiguous bindings, ensuring the uniqueness, stability, and high reliability of the final mapping relationship, and avoiding confusion or misuse during subsequent error correction. The establishment of a dual mapping relationship enables each OCR misidentified item to have a clear named semantic label and structural feature pointing, which can serve as a direct basis for replacing error correction candidates, greatly improving the automation and semantic rationality of text error correction.

[0041] Preferably, step S3, which utilizes the dual mapping relationship between the first and second data naming to automatically correct and re-identify OCR misidentified candidates, includes:

[0042] Using a dual mapping relationship, semantic replacement matching of incorrectly identified OCR candidate options is performed, successfully matched candidate options and their suggested replacements are extracted, and error correction matching data is generated.

[0043] Based on the error correction matching data, target character replacement is performed at the corresponding position in the electronic file to be processed to generate preliminary corrected text data;

[0044] Contextual language validation is performed on the initially corrected text data to determine semantic coherence, resulting in incoherent segments. These incoherent segments are then rewritten using probability-optimal methods to generate context-consistent corrected data.

[0045] The initial corrected text is updated based on the context consistency correction data to form the corrected and accurate text data.

[0046] This invention utilizes a pre-established dual naming mapping relationship to semantically match OCR misidentified candidates with preset named entities, automatically extracting highly matched suggested replacements. This achieves precise error correction entry point judgment through "semantic-driven + structural alignment," effectively avoiding erroneous replacements and missed corrections. Based on the error correction matching data, it directly locates the position of the original characters in the image or text of the electronic document and performs target character replacement operations, generating preliminary corrected text data. This ensures that each replacement operation is traceable, controllable, and formatted consistently, making it particularly suitable for error correction of tables and field-type information. The pre-corrected text undergoes contextual language verification to identify semantically incoherent sentences or logically anomalous context fragments, preventing single-point character replacements from causing overall semantic ambiguity or expression errors, achieving dual protection of semantic integrity and syntactic consistency. For incoherent fragments, the system employs an optimal candidate generation strategy based on language models and contextual probabilities for automatic rewriting, ensuring that the corrected sentences are semantically coherent, logically fluent, and linguistically natural, significantly improving overall text quality and natural language readability. The entire process forms a closed-loop error correction mechanism of "mapping matching → replacement application → language verification → optimization rewriting", which ensures that the final output text is not only accurate at the character level, but also consistent at the semantic, syntactic and sentence structure levels, significantly improving the overall confidence and semantic accuracy of structured data.

[0047] Preferably, performing contextual language validation on the initially revised text data to determine semantic coherence and identify incoherent segments includes:

[0048] Syntactic structure analysis was performed on the preliminary revised text data to extract the grammatical dependency matrix of the text. The sentence length range was set between 5 and 40 words, and the part exceeding the range was divided into multiple clauses according to the natural semantic boundaries.

[0049] For each text clause, a contextual coherence score is calculated, with the coherence score ranging from 0 to 1. Bidirectional prediction probabilities are calculated for the current sentence and the sentences before and after it. When the joint prediction probability between sentences is less than 0.35, it is initially marked as a suspicious breakpoint.

[0050] By introducing semantic nesting relationships and entity continuity analysis, if the subject entities of two consecutive clauses lack logical connection or there is an obvious verb-object collocation error, they are marked as paragraphs with abnormal semantic structure.

[0051] Context window resampling verification is performed on paragraphs with semantic structure abnormalities based on all suspicious breakpoints. The window size is 2 sentences before and after the paragraph. If the coherence score is less than 0.4 in more than 3 out of 5 samples, the paragraph is finally confirmed as a disjoint segment.

[0052] This invention performs syntactic structure analysis on the initially revised text, extracts the dependency matrix, and sets a reasonable sentence length range (5-40 words). Longer content is automatically segmented based on natural semantic boundaries, effectively handling complex expressions such as long sentences, comma-connected sentences, and nested phrases, ensuring accurate structural input for subsequent semantic judgment. It performs bidirectional context prediction and joint probability calculation on each clause and its preceding and following statements, marking probabilities below 0.35 as potential semantic breakpoints. This overcomes the limitations of traditional language models that rely solely on local grammatical judgment, comprehensively improving the ability to perceive contextual logical relationships. Based on semantic nesting and logical continuity analysis of core entities such as subjects, objects, and action verbs, it determines whether semantic errors exist between clauses, such as subject abrupt changes, verb-object mismatches, and logical jumps in argumentation. It moves from the language expression layer to the semantic understanding layer, effectively identifying grammatically correct but semantically broken abnormal segments. A sliding window strategy with two sentences before and after the breakpoint is used to resample the suspected breakpoint. Based on five independent context judgments, if the coherence score is less than 0.4 three or more times, it is finally confirmed as a disjointed segment, which improves the robustness of the detection process to texts with ambiguous boundaries and complex contexts.

[0053] Preferably, step S4 includes the following steps:

[0054] Step S41: Segment the corrected and accurate text data, extract semantic paragraph ranges, and generate structured paragraph boundary data;

[0055] Step S42: Perform template field keyword matching on each paragraph area based on the structured paragraph boundary data, extract candidate fragments of the matching fields, and generate candidate content data for the fields;

[0056] Step S43: Perform semantic normalization on the candidate content data of the field, remove redundant symbols and unify the unit format to generate standardized data of the field;

[0057] Step S44: Map the standardized field data to the preset information field template, extract the corresponding field values, and generate structured information field data; perform field integrity verification on the structured information field data, remove missing key fields or abnormal field values, and output structured information.

[0058] The present invention performs semantic paragraph recognition on corrected accurate text data to generate structured paragraph boundary data, achieving logical segmentation of document content at the semantic level, avoiding static segmentation errors based on line numbers or page numbers, and improving the positioning accuracy and stability of key fields in natural language paragraphs. Keyword matching is performed on each paragraph area according to a preset field template to extract candidate content containing target fields, reducing interference from irrelevant text and enhancing the recall rate and processing efficiency of field extraction, which is particularly suitable for file types where the position or word order of fields is not fixed. Semantic normalization operations are performed on the candidate field content, including unit conversion, numerical standardization, redundant symbol removal, and format unification (such as converting "Three thousand yuan" to "3000 yuan"), to solve the problem of chaotic data expression caused by OCR misrecognition, writing differences, diverse language styles, etc., and improve the consistency and comparability of field values. By establishing a one-to-one mapping relationship between the standardized field data and the information field template, the corresponding structure of field names and field values is accurately extracted, effectively supporting subsequent structured operations such as data form generation, database storage, or knowledge graph construction. Integrity verification is performed on the structured field results, including operations such as judging the absence of key fields, verifying the legality of field value formats, and filtering abnormal fields, to ensure that the output data has high integrity, correctness, and usability, significantly improving the usage effect of downstream systems and the level of data quality assurance.

[0059] In this specification, an information extraction system for electronic files is provided, which is used to execute the information extraction method for electronic files described above. The information extraction system for electronic files includes:

[0060] A misrecognition detection module, which is used to obtain the to-be-processed electronic file and perform OCR recognition to generate initial text data; perform error detection on the initial text data to identify OCR misrecognition candidates therein;

[0061] A data naming module, which is used to generate a first data name for each OCR misrecognition candidate according to its context semantics and layout structure; extract low-level features of the first data name, and use a preset domain word list to perform named entity recognition on each OCR misrecognition candidate in combination with the first data name, so as to assign a second data name;

[0062] An automatic correction module, which is used to perform automatic error correction and re-recognition on the OCR misrecognition candidates by using the dual mapping relationship between the first data name and the second data name, and generate corrected accurate text data;

[0063] An information extraction module, which is used to extract key information from the corrected accurate text data according to a preset information field template and output structured information.

[0064] The beneficial effects of this invention are as follows: The misidentification detection module can perform confidence analysis and semantic integrity verification on the initial OCR text, intelligently identify potential recognition error areas, significantly improve the accuracy of misidentification location, reduce the burden of manual review, and effectively adapt to recognition difficulties such as uneven quality or complex layout of archival images. The data naming module uses contextual semantic analysis and layout structure extraction in parallel to construct the first data name, and combines a preset domain vocabulary to perform named entity recognition to generate the second data name, overcoming the problem of weak field semantic understanding ability of traditional OCR systems and realizing a semantic-driven naming generation mechanism. The automatic correction module constructs a semantic + structural mapping relationship between the first and second data names, performs semantic replacement and context rewriting on misidentified items, and can effectively correct common errors such as character replacement, missing characters, misplacement, and homophony misidentification, significantly improving the overall text quality and semantic accuracy. The information extraction module can perform candidate content recognition, field normalization, and structural mapping based on field templates to ensure that the output information fields have high accuracy, high integrity, and good database integration capabilities, meeting the standardized data input requirements of various business systems. Therefore, this invention solves the core technical bottlenecks of existing OCR, such as inaccurate recognition, poor error correction capability, and low intelligence of information extraction, through a four-in-one process of "misidentification detection + named entity recognition + semantic error correction + template extraction", and significantly improves the accuracy, stability and intelligence level of electronic archive information extraction. Attached Figure Description

[0065] Figure 1 A flowchart illustrating the steps of an information extraction method for electronic archives;

[0066] Figure 2 for Figure 1 A detailed flowchart illustrating the implementation steps of step S1.

[0067] Figure 3 for Figure 1 A detailed flowchart illustrating the implementation steps of step S4.

[0068] Figure 4 A schematic diagram illustrating an information extraction method for electronic archives;

[0069] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0070] The technical method of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0071] Furthermore, the accompanying drawings are merely illustrative of the invention and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor methods and / or microcontroller methods.

[0072] It should be understood that although the terms "first," "second," etc., may be used herein to describe various units, these units should not be limited by these terms. These terms are used merely to distinguish one unit from another. For example, without departing from the scope of the exemplary embodiments, a first unit may be referred to as a second unit, and similarly, a second unit may be referred to as a first unit. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0073] To achieve the above objectives, please refer to Figures 1 to 4 A method for extracting information from electronic archives, the method comprising the following steps:

[0074] Step S1: Obtain the electronic document to be processed and perform OCR recognition to generate initial text data; perform error detection on the initial text data to identify OCR misrecognition candidates;

[0075] Step S2: For each OCR misidentification candidate, generate a first data name based on its contextual semantics and layout structure; extract the low-level features of the first data name, and use a preset domain vocabulary in combination with the first data name to perform named entity recognition on each OCR misidentification candidate, thereby assigning a second data name.

[0076] Step S3: Utilize the dual mapping relationship between the first data naming and the second data naming to automatically correct and re-identify the OCR misidentified candidates, generating corrected and accurate text data;

[0077] Step S4: Extract key information from the corrected and accurate text data according to the preset information field template, and output structured information.

[0078] This invention effectively corrects the recognition errors of conventional OCR systems for similar characters, ambiguous text, and industry-specific terms by performing contextual semantic parsing and layout structure analysis on misidentified candidates in the initial OCR text, combined with a first data naming generation rule and a second data naming mapping strategy. This significantly improves the text recognition accuracy of electronic documents. Utilizing a dual mechanism of first data naming (based on context and layout structure) and second data naming (combined with semantic naming based on a domain thesaurus), the system achieves high-precision recognition and semantic reconstruction of key information entities such as names of people, places, organizations, times, and values, enhancing the semantic integrity of the text. The system not only detects significant recognition errors but also performs deep error correction analysis on words with similar structures but inconsistent meanings, possessing a certain degree of semantic fault tolerance. It is suitable for error correction scenarios in complex contexts, such as high-density professional texts like historical archives, official documents, and medical reports. By introducing configurable preset domain thesaurus and template libraries, the method can quickly adapt to the electronic document processing needs of different industries (such as legal, medical, engineering, and financial sectors), exhibiting good scalability and industry universality. By introducing a structured template extraction mechanism in the OCR post-processing stage, the identified and corrected text can be extracted according to field logic to output structured data, providing standardized data support for subsequent archiving management, retrieval analysis, and knowledge graph construction. This method automates the entire process of identification, error correction, naming, and structure extraction, eliminating the need for manual verification and modification, significantly reducing labor costs and processing time, and is particularly suitable for large-scale automated document processing tasks. Through semantic-driven OCR error correction and field reconstruction, the generated text is closer to the original semantics and format, enhancing the clarity, standardization, and retrieval value of document data in subsequent use. Therefore, this invention, through a four-in-one process of "misidentification detection + named entity recognition + semantic error correction + template extraction," solves the core technical bottlenecks of existing OCR technologies, such as inaccurate identification, poor error correction capabilities, and low information extraction intelligence, significantly improving the accuracy, stability, and intelligence level of electronic archive information extraction.

[0079] In this embodiment of the invention, reference Figure 1 and Figure 4 The diagram shown is a flowchart illustrating the steps of an information extraction method for electronic archives according to the present invention. In this example, the information extraction method for electronic archives includes the following steps:

[0080] Step S1: Obtain the electronic document to be processed and perform OCR recognition to generate initial text data; perform error detection on the initial text data to identify OCR misrecognition candidates;

[0081] Step S2: For each OCR misidentification candidate, generate a first data name based on its contextual semantics and layout structure; extract the low-level features of the first data name, and use a preset domain vocabulary in combination with the first data name to perform named entity recognition on each OCR misidentification candidate, thereby assigning a second data name.

[0082] Step S3: Utilize the dual mapping relationship between the first data naming and the second data naming to automatically correct and re-identify the OCR misidentified candidates, generating corrected and accurate text data;

[0083] Step S4: Extract key information from the corrected and accurate text data according to the preset information field template, and output structured information.

[0084] In this embodiment of the invention, an electronic document file to be processed is acquired, supporting multiple formats such as PDF and scanned images. A high-performance OCR recognition engine is used to perform text recognition on the electronic document, extracting text information and generating initial text data. Subsequently, error detection is performed on the initial text data. Rule-based syntax verification and statistical language models are used to analyze the recognized text, identifying candidate items with misidentification. These candidate items typically exhibit character anomalies, word mismatches, or semantically unreasonable positions. For each identified OCR misidentified candidate item, a first data name is generated based on the contextual semantic information and layout structure of the text containing the candidate item (such as paragraph position, font format, row and column relationships), serving as the preliminary classification label for that candidate item. Low-level features are extracted from the first data name, including character length, character type, and adjacent character relationships, and compared with a pre-established domain lexicon. Named entity recognition technology (such as Conditional Random Field (CRF) or deep learning models) is used to reassign a second data name to the candidate item, ensuring accurate classification of the candidate item at the domain-specific semantic level. A dual mapping relationship is established between the first and second data names as the basis for automatic error correction. This process utilizes mapping relationships to perform error correction and re-identification operations on OCR misidentified candidates. Context-based dictionary matching and similarity calculation methods are employed to correct the recognition results and improve text accuracy. This process generates corrected, accurate text data, ensuring semantic rationality and standardized formatting. Based on the corrected, accurate text data, key information is extracted according to pre-defined information field templates (including field names, data types, and field positions). Regular expressions and template matching methods are used to locate and extract the required field data. Finally, the extracted key information is output in a structured format, such as JSON, XML, or database format, facilitating subsequent data processing and application.

[0085] As an example of the present invention, reference is made to... Figure 2 As shown, in this example, step S1 includes:

[0086] Step S11: Obtain the electronic file to be processed;

[0087] Step S12: Perform image analysis and region division on the electronic archives to be processed, extract structured text image regions, and obtain image partition data;

[0088] Step S13: Perform OCR recognition on the image partition data to extract the text content in the electronic document and generate initial text data; perform character confidence analysis on the initial text data to extract the positions of characters with recognition confidence below a preset threshold and generate low-confidence character annotation data;

[0089] Step S14: Combine the initial text data with the low-confidence character annotation data, perform semantic integrity analysis, identify segments with semantic missing or logical interruptions, and generate semantic anomaly candidate segment data; perform intersection location on the low-confidence character annotation data and the semantic anomaly candidate segment data, annotate the OCR misidentification area, and generate OCR misidentification candidate data.

[0090] In this embodiment of the invention, an electronic document to be processed is acquired. This electronic document can be a scanned file, PDF format, or other digital image format. The electronic document is imported into the processing system through a file reading interface to ensure data integrity and file format compatibility. For the electronic document to be processed, image processing algorithms are used to perform image analysis. This process includes image preprocessing (such as denoising, binarization, and grayscale transformation) and layout analysis techniques. Region partitioning algorithms (such as projection analysis, connected component analysis, and deep learning-based layout detection) are used to divide the electronic document into several structured text image regions. Each region contains relatively independent text content or mixed text and images, generating structured image partition data, indicating the coordinate information and content type of the region. An OCR recognition engine is applied to the image partition data to extract text information from the image content within each partition, generating initial text data. During the OCR process, a character recognition algorithm (such as a convolutional neural network CNN) is used to obtain the confidence score of each character. Character positions with confidence levels below a preset threshold (e.g., 0.75) are labeled to generate low-confidence character labeling data. This data records the character's image position, confidence value, and corresponding text position for subsequent processing. Semantic integrity analysis is performed by combining the initial text data and the low-confidence character labeling data. Natural language processing techniques (such as language model-based syntactic analysis and semantic coherence detection) are used to identify segments in the text containing semantic missing meanings, logical breaks, or semantic contradictions, forming candidate semantic anomaly segments. Simultaneously, the spatial and textual intersection operations of the low-confidence character labeling data and the candidate semantic anomaly segments are performed to accurately locate suspected OCR misidentification areas. Based on the intersection localization results, candidate OCR misidentification data is labeled for subsequent automatic error correction and text correction.

[0091] Preferably, in step S13, OCR recognition is performed on the image partition data to extract the text content from the electronic document, including:

[0092] The image partitioning data is divided into image partitioning data blocks with a unit size of 128×128 pixels. Edge detection and connected component analysis are performed on each partition, and connected components with an area of ​​not less than 200 pixels are selected. 2 Suspected text areas with aspect ratios between 2:1 and 10:1;

[0093] Perform character-level OCR recognition in each suspected area. It supports recognizing characters with sizes ranging from 12pt to 36pt. For characters smaller than 10pt, double pixel interpolation is performed before recognition to obtain the recognized character information.

[0094] The identified character information is evaluated for confidence and marked as anomalies. Characters with a confidence level of less than 85% are marked as low-confidence characters. When the average confidence level of the entire text is higher than 92% and the recognition error rate is lower than 3%, it is marked as a valid text segment.

[0095] Based on the valid text segments, the original coordinates of the electronic file to be processed are mapped and the position is reconstructed. The images are then stitched and sorted according to the order from top to bottom and from left to right, and the output is structured initial text data with field numbers, character content, position information, character spacing and average confidence.

[0096] In this embodiment of the invention, the input image partition data is divided into data blocks, with the entire image region divided into multiple small units of a fixed size of 128 pixels by 128 pixels. Within each small unit, image edges are detected using an edge detection algorithm (such as the Canny operator). Subsequently, connected component analysis is applied to determine the connectivity of the detected edge regions, filtering out connected components with an area of ​​not less than 200 square pixels and an aspect ratio between 2:1 and 10:1 as suspected text regions. For the filtered suspected text regions, character-level OCR recognition technology is used for text extraction. The recognition system supports recognizing text content with character sizes ranging from 12 to 36 points. For characters smaller than 10 points, the corresponding image region is first enlarged by double pixel interpolation to improve image resolution before OCR recognition, thereby ensuring the accuracy of small character recognition. The OCR engine, based on convolutional neural networks and deep learning models, extracts text information character by character and returns the corresponding recognition results. After recognition, a confidence assessment is performed on each character. Characters with a single-character recognition confidence level below 85% are marked as low-confidence characters for subsequent error correction. Simultaneously, the average confidence level and recognition error rate are calculated for the entire text segment. When the average confidence level reaches or exceeds 92% and the recognition error rate is controlled below 3%, the entire text segment is marked as a valid text segment, indicating that the recognition result of that segment is relatively reliable. Finally, based on the text order of the valid text segments and the original coordinates of the image partitions, the original image coordinates are mapped and the positions of the text are reconstructed. The recognized text segments are concatenated and sorted in the image from top to bottom and from left to right to construct structured initial text data with field numbers, character content, character position information (including coordinates and character spacing), and average character confidence levels, for subsequent text analysis and error correction.

[0097] Preferably, in step S2, for each OCR misidentification candidate, generating a first data name based on its contextual semantics and layout structure includes:

[0098] For each OCR misidentified candidate, extract the corresponding preceding and following text lines to generate context text segment data;

[0099] Perform part-of-speech tagging and syntactic structure analysis on the context text segment data, extract sentence structure and keyword sequences, and generate semantic analysis result data;

[0100] By matching the OCR misidentification candidate data with the semantic analysis results data, and matching the field names or common phrases in the standard terminology database, semantic matching candidate field data is generated.

[0101] Extract the relative position, alignment mode, and font style of the OCR misidentified candidate data in the image partition data to generate page layout feature data;

[0102] The system integrates semantic matching candidate field data with layout structure feature data, performs context consistency scoring, and generates the first data name based on the highest score result.

[0103] In this embodiment of the invention, for each OCR misidentified candidate, contextual information is first extracted from the text stream in which it resides. Specifically, by locating the position of the candidate in the text sequence, several lines before and after the line containing the candidate (e.g., two lines before and after) are extracted to form contextual text segment data. This contextual text segment contains words and sentences surrounding the candidate, providing a complete linguistic environment for subsequent semantic analysis. Next, part-of-speech tagging and syntactic structure analysis are performed on the obtained contextual text segment data. Natural language processing toolkits (such as part-of-speech taggers based on conditional random fields or deep learning) are used to assign part-of-speech values ​​to each word in the context, identifying categories such as nouns, verbs, and adjectives. Simultaneously, a syntactic analyzer is used to parse sentence components, extract sentence structure (subject, verb, object, etc.) and key modification relationships, generating semantic analysis result data. This data contains keyword sequences and their syntactic relationships, providing a basis for subsequent field identification. Subsequently, the OCR misidentified candidate data itself is matched with the above semantic analysis results. Specifically, the candidate is compared with a pre-established standard terminology database, which contains commonly used field names, professional terms, and phrases in the domain. String similarity calculation methods (such as edit distance or cosine similarity) are used to match candidate options and their context keywords with entries in a terminology database, generating semantic matching candidate field data and filtering out several corresponding field names. Simultaneously, based on the position of OCR misidentified candidate options within the image partitions, layout structure feature data is extracted. This includes the relative position coordinates of the candidate options, their alignment with surrounding text (left alignment, center alignment, right alignment), and font style information (font type, size, weight, italics, etc.). This information is obtained through image processing and text layout parsing techniques, providing a layout basis for determining the candidate option field type. Finally, the semantic matching candidate field data and layout structure feature data are fused. By setting a context consistency scoring mechanism, a comprehensive score for the candidate fields is calculated by combining the semantic matching score and the layout feature matching degree. Based on the result of the highest comprehensive score, the final first data name is determined, completing the field identification of OCR misidentified candidate options for subsequent error correction and data integration.

[0104] Preferably, step S2 involves extracting low-level features from the first data naming and using a preset domain vocabulary in conjunction with the first data naming to perform named entity recognition on each OCR misidentification candidate, including:

[0105] The first data name is subjected to character-level encoding and sub-word segmentation to extract its low-level language features such as character length, stem structure and naming format, and generate naming low-level feature data;

[0106] The system uses named low-level feature data to match pre-defined domain terms and generates candidate domain entity label data based on rules.

[0107] The first data name and candidate domain entity label data are used to perform named entity classification scoring to identify misclassified or ambiguous entity ranges and generate entity confidence score data.

[0108] Based on the entity confidence score data, the final entity type is determined for each OCR misidentification candidate, and named entity label binding is performed to generate a second data name.

[0109] In this embodiment of the invention, the generated first data name is processed by character-level encoding, breaking it down into basic character units. Further, sub-word segmentation techniques (e.g., based on Byte Pair Encoding or WordPiece algorithms) are used to segment the string, extracting low-level language features such as character length, stem structure, and common naming formats. These features include character type distribution (letters, numbers, symbols), root and affix structure, and naming conventions (e.g., camelCase, underscore separation, numeric suffixes), generating detailed low-level naming feature data. Subsequently, this low-level naming feature data is used for matching within a pre-defined domain thesaurus. The domain thesaurus contains common professional terms, keywords, and entity names within the domain. Candidate terms are selected using fuzzy matching, partial matching, and other rules, combined with low-level features. Matching rules include tolerance for word form transformations, synonym replacement, and character edit distance limitations, generating candidate domain entity label data and labeling the corresponding entity types, such as personal names, place names, organization names, and product models. Finally, the first data name and the candidate domain entity label data are compared for named entity classification and scoring. A rule-based scoring mechanism or machine learning-assisted algorithm is employed to evaluate the accuracy of matching and the confidence level of entity attribution. This scoring considers the contextual consistency of entity names, dictionary matching accuracy, and low-level feature matching precision to identify potential misclassifications and ambiguous entity boundaries, generating entity confidence score data. Finally, based on the entity confidence score data, the final entity type is determined for each OCR misidentification candidate. Through threshold filtering and multi-dimensional confidence weighting, a named entity label binding operation is performed, binding appropriate entity labels to candidate items to generate a second data naming with clear semantics. After this process, the second data naming dataset provides the foundation for subsequent automatic error correction and information extraction.

[0110] Preferably, the method for confirming the dual mapping relationship between the first data naming and the second data naming in step S3 includes:

[0111] The first and second data names are compared by position index, and the relative position mapping information of the corresponding items in the original text is extracted to generate named position mapping data.

[0112] Based on the naming position mapping data, a similarity measurement is performed on the character content of the first data name and the second data name to generate naming semantic similarity data;

[0113] Based on the naming semantic similarity data, name pairs with similarity higher than a preset threshold are selected, an initial bidirectional naming mapping table is constructed, and a candidate dataset for naming mapping is generated.

[0114] Conflict checking and ambiguity alignment are performed on the candidate datasets for naming mapping, and ambiguous naming bindings are filtered out, thereby obtaining a dual mapping relationship between the first and second data names.

[0115] In this embodiment of the invention, a position index comparison is performed between the first and second data names. Specifically, the start and end positions of the characters in the original text data are extracted, and relative position mapping information is calculated to form a position index lookup table. This process utilizes text character offset positioning technology to ensure accurate matching of different naming identifiers of the same text segment, generating detailed naming position mapping data. Subsequently, based on the naming position mapping data, character-level content similarity measurement is performed on the first and second data names at corresponding positions. Multiple similarity algorithms, such as edit distance, Jaccard coefficient, and word vector cosine similarity, are used to comprehensively score the named text, forming naming semantic similarity data. This step not only compares character surface matching but also incorporates semantic similarity evaluation to improve semantic matching accuracy. Next, based on the above semantic similarity data, naming pairs with similarity exceeding a preset threshold (e.g., 0.85) are selected to construct an initial bidirectional naming mapping table. This mapping table records all correspondences between the first and second data names that meet the similarity requirements, forming a naming mapping candidate dataset. This process ensures complete coverage of high-confidence naming pairs, facilitating subsequent processing. Finally, conflict resolution and ambiguity alignment are performed on the candidate naming mapping datasets. By analyzing one-to-many, many-to-one, and many-to-many relationships in the mapping table, and combining contextual semantics and layout structure information, ambiguous or conflicting naming bindings are identified and filtered. Rule-based filtering and semantic consistency checks are used to eliminate ambiguous or duplicate mapping items, thereby ultimately confirming the unique and accurate dual mapping relationship between the first and second data naming, ensuring the uniqueness and accuracy of the mapping relationship.

[0116] Of particular importance, performing a similarity measurement on the character content of the first and second data names based on the naming location mapping data also includes:

[0117] Parse the location information of the named location mapping data, extract the corresponding character content segments of each group of first and second data names, and generate named pair character extraction data;

[0118] Perform word segmentation and part-of-speech tagging on each pair of characters in the named character extraction data to generate named character semantic unit data;

[0119] Using named character semantic unit data, a context-weighted word vector model is called to construct semantic vectors for the naming of the first data and the naming of the second data respectively, generating named semantic vector data;

[0120] For each named pair in the named semantic vector data, perform cosine similarity calculation to generate preliminary semantic similarity score data;

[0121] The initial semantic similarity score data is standardized by applying a preset semantic similarity threshold to output named semantic similarity data.

[0122] In this embodiment of the invention, the named position mapping data is parsed for positional information. This data records the start and end character indices of the first and second named data in the initial text. By parsing each named position pair, its character content in the structured initial text data is extracted, forming named pair character extraction data. This data structure is stored in key-value pair format, with each record containing the character fragment of the first named data, the character fragment of the second named data, and its corresponding character position index. Next, Chinese word segmentation and part-of-speech tagging are performed on the character content in the named pair character extraction data. Using a Chinese natural language processing tool based on CRF (Conditional Random Field) or BERT-BiLSTM-CRF structure, word boundaries are divided for each named character fragment, and the part-of-speech tagging of each word (such as noun, verb, numeral, proper noun, etc.) is assigned. This operation generates named character semantic unit data, where each record includes a word item, part-of-speech tag, and its position information in the fragment, which is used to construct semantic vectors later. Then, using the named character semantic unit data, a word vector model based on context weights is called to construct semantic vectors. The word vector model employs a context-sensitive model (such as BERT, ERNIE, or RoFormer) trained on Chinese corpora to encode the context-related semantic information of each word into a high-dimensional vector. Semantic vectors are constructed using average pooling or weighted aggregation for both the first and second data names, forming named semantic vector data. Each naming pair generates two vector representations, representing the overall semantic information of the two named text fragments. Subsequently, cosine similarity is calculated for each naming pair in the named semantic vector data. Using a standard cosine similarity calculation method, the angle between the first and second data naming vectors is quantified, yielding preliminary semantic similarity scores ranging from [-1, 1]. Generally, scores below 0 indicate semantic contradiction, while scores close to 1 indicate high similarity. Finally, the preliminary semantic similarity scores are standardized using a preset semantic similarity threshold. All scores are uniformly mapped to the range [0, 100], and a threshold (e.g., 70 points) is set to determine whether the naming pairs meet the semantic equivalence standard. If the score is higher than the threshold, it is determined that there is a valid semantic mapping relationship between the first data name and the second data name. Finally, the semantic similarity data of the names is output, including the original score, the standardized scoring result and the threshold judgment mark, which are used for subsequent error correction and matching logic judgment.

[0123] Preferably, step S3, which utilizes the dual mapping relationship between the first and second data naming to automatically correct and re-identify OCR misidentified candidates, includes:

[0124] Using a dual mapping relationship, semantic replacement matching of incorrectly identified OCR candidate options is performed, successfully matched candidate options and their suggested replacements are extracted, and error correction matching data is generated.

[0125] Based on the error correction matching data, target character replacement is performed at the corresponding position in the electronic file to be processed to generate preliminary corrected text data;

[0126] Contextual language validation is performed on the initially corrected text data to determine semantic coherence, resulting in incoherent segments. These incoherent segments are then rewritten using probability-optimal methods to generate context-consistent corrected data.

[0127] The initial corrected text is updated based on the context consistency correction data to form the corrected and accurate text data.

[0128] In this embodiment of the invention, semantic replacement matching is performed on OCR misidentified candidate items in the electronic document to be processed through a dual mapping relationship established based on the first data naming and the second data naming. Specifically, by comparing the candidate item text with the second data naming in the mapping relationship, string similarity algorithms (such as edit distance and word vector similarity) and contextual semantic matching technology are used to extract all successfully matched candidate items and their corresponding suggested replacements, generating error correction matching data. This matching process strictly limits the character length and semantic category of the replacement candidate items to ensure the accuracy and rationality of the replacement suggestions. Subsequently, the error correction matching data is used to perform target character replacement operations on the candidate items at the corresponding positions in the electronic document text. During the replacement process, the misidentified candidate items are located according to the character start and end indexes, and they are replaced word by word according to the suggested replacement content to form preliminary corrected text data. After the text replacement, character spacing and format preservation processing are performed to ensure that the text structure is consistent with the original document. Next, contextual language verification is performed on the preliminary corrected text data. Specifically, a semantic coherence detection method based on statistical language models or deep learning language models is used to analyze the linguistic logic and semantic consistency of continuous sentence segments in the text, and to identify text fragments that are semantically incoherent or ambiguous. For disjointed segments, a probabilistic optimal rewriting technique is used for local text rewriting, including lexical replacement, phrase recombination, and syntactic adjustments, to generate context-consistency corrected data. Finally, based on the context-consistency corrected data, the initially corrected text is updated and replaced to correct semantic incoherence anomalies, achieving accurate overall contextual coherence. Upon completion, the corrected and accurate text data is output, providing a reliable foundation for subsequent structured information extraction.

[0129] Preferably, performing contextual language validation on the initially revised text data to determine semantic coherence and identify incoherent segments includes:

[0130] Syntactic structure analysis was performed on the preliminary revised text data to extract the grammatical dependency matrix of the text. The sentence length range was set between 5 and 40 words, and the part exceeding the range was divided into multiple clauses according to the natural semantic boundaries.

[0131] For each text clause, a contextual coherence score is calculated, with the coherence score ranging from 0 to 1. Bidirectional prediction probabilities are calculated for the current sentence and the sentences before and after it. When the joint prediction probability between sentences is less than 0.35, it is initially marked as a suspicious breakpoint.

[0132] By introducing semantic nesting relationships and entity continuity analysis, if the subject entities of two consecutive clauses lack logical connection or there is an obvious verb-object collocation error, they are marked as paragraphs with abnormal semantic structure.

[0133] Context window resampling verification is performed on paragraphs with semantic structure abnormalities based on all suspicious breakpoints. The window size is 2 sentences before and after the paragraph. If the coherence score is less than 0.4 in more than 3 out of 5 samples, the paragraph is finally confirmed as a disjoint segment.

[0134] In this embodiment of the invention, syntactic structure analysis is performed on the initially corrected text data. A natural language processing tool based on dependency parsing is used to construct a syntactic dependency relation matrix for each sentence in the text. Sentence length is limited to between 5 and 40 words. If a sentence exceeds 40 words, it is segmented into multiple reasonable clauses based on natural semantic boundaries (such as commas, semicolons, conjunctions, etc.) to ensure the accuracy and computational efficiency of subsequent analysis. Next, for each segmented text clause, its contextual coherence score is calculated. This score is calculated bidirectionally using a statistical language model or a deep learning language model, predicting the semantic matching probability of adjacent clauses based on the content of the current clause. The coherence score is set between 0 and 1. If the joint prediction probability of the current sentence and its preceding and following clauses is less than 0.35, the sentence is initially marked as a suspicious breakpoint, indicating semantic incoherence. Subsequently, semantic nesting relationships and entity continuity analysis are further introduced. By constructing a core entity chain in the text, the logical connection between the subject entities of two consecutive clauses is detected. If a lack of subject coherence is found, or if there is a significant verb-object mismatch between two clauses (such as a subject-verb mismatch), the text region is marked as a semantically structurally abnormal segment and is subject to further investigation. Finally, for all marked suspicious breakpoints, a context-window-based resampling verification is performed. The window size is set to two sentences before and after the current breakpoint, and multiple (e.g., five) random context resampling checks are conducted. If, in these five samples, the context coherence score is below 0.4 for more than three samples, the text segment is ultimately confirmed as an incoherent fragment. This incoherent fragment is then output for subsequent semantic correction and text reconstruction steps.

[0135] As an example of the present invention, reference is made to... Figure 3 As shown, step S4 in this example includes:

[0136] Step S41: Segment the corrected and accurate text data, extract semantic paragraph ranges, and generate structured paragraph boundary data;

[0137] Step S42: Perform template field keyword matching on each paragraph area based on the structured paragraph boundary data, extract candidate fragments of the matching fields, and generate candidate content data for the fields;

[0138] Step S43: Perform semantic normalization on the candidate content data of the field, remove redundant symbols and unify the unit format to generate standardized data of the field;

[0139] Step S44: Map the standardized field data to the preset information field template, extract the corresponding field values, and generate structured information field data; perform field integrity verification on the structured information field data, remove missing key fields or abnormal field values, and output structured information.

[0140] In this embodiment of the invention, the corrected and accurate text data is segmented. A paragraph boundary detection algorithm from natural language processing is used to identify semantic paragraph boundaries in the text based on line breaks, blank lines, punctuation marks, and contextual semantic pauses. Paragraph lengths are generally controlled between 50 and 300 characters to ensure the integrity and logical coherence of the paragraph content. Text indexing and positioning technology is used to extract the start and end character positions of each paragraph, generating structured paragraph boundary data. This data includes the paragraph number, start position, and end position, facilitating subsequent accurate location and retrieval. Based on the structured paragraph boundary data, template field keyword matching is performed on the text within each paragraph. A preset keyword library, including field name synonyms, hyponyms, and related expressions, is used, combined with regular expression technology and fuzzy matching algorithms, to locate the occurrence of keywords in the paragraph text. Text content within a certain range (e.g., within 50 characters before and after) of the keywords is extracted to generate candidate content data. During keyword matching, a matching threshold is set; for example, keywords with an edit distance of no more than 2 are considered a successful match, ensuring accurate coverage of candidate content. Semantic normalization is performed on candidate field content data. First, text cleaning techniques are used to remove special symbols, meaningless characters, and redundant spaces from the candidate content. Then, unit conversion rules are combined to unify the unit format, such as unifying "kilogram" and "kg" to "kg", and time format to "YYYY-MM-DD", etc. For numerical fields, number standardization methods are used to unify the precision and format of numerical expressions. After normalization, standardized field data is output to ensure consistency in format and semantics among field content from different sources. The standardized field data is mapped to a preset information field template. Based on the field names and data types defined in the template, the standardized field content is mapped to each field item, and the corresponding field values ​​are extracted. Field integrity verification rules are used to determine whether a field is missing or whether the field value exceeds a reasonable range. For example, whether the date field conforms to the time range, and whether the numerical field is within the allowed range. Field values ​​that do not meet the integrity requirements or are abnormal are removed, finally generating structured information field data that meets the template requirements. This data has a unified format, facilitating subsequent storage, analysis, and retrieval.

[0141] Of particular importance, the process of performing template field keyword matching on each paragraph region based on structured paragraph boundary data and extracting candidate segments for matching fields also includes:

[0142] Perform paragraph numbering on the structured paragraph boundary data, mark the location information of each paragraph area, and generate paragraph index annotation data;

[0143] Using paragraph index annotation data, the preset field keyword templates are called sequentially for each paragraph area to perform keyword group Boolean matching and generate field keyword hit data;

[0144] Extract the hit position from the keyword hit data of the field, record the paragraph where the matched field is located and the relative position of the keyword, and generate candidate data for the field position;

[0145] Based on the candidate data of the field position, perform phrase extraction operation on the adjacent context in the hit paragraph, extract the local text fragment containing the field content, and generate the original fragment data of the field;

[0146] Perform noise character removal on the original fragment data of the field to remove redundant punctuation, newline characters or garbled characters, and output the candidate content data of the field.

[0147] In the embodiments of the present invention, paragraph numbering processing is performed on the structured paragraph boundary data. Each paragraph is assigned a unique number in the order of its appearance in the original text, starting from "P001" and numbered sequentially. At the same time, the start and end character position indexes of the paragraph in the text are recorded. This operation is achieved by programmatically traversing the text paragraph segmentation results. The output paragraph index annotation data is saved in a structured form. Each record contains the paragraph number, start character index, end character index, and the original paragraph content, providing precise positioning support for subsequent field matching. Subsequently, using the paragraph index annotation data, a preset field keyword template is called for each paragraph region. The field keyword template consists of a field name and the corresponding keyword group. Each group of keywords contains a core keyword and extended synonyms. The matching adopts a Boolean logic method. For example, the keyword group of the "contract number" field contains "contract number", "contract ID", "number", etc. It is judged whether any keyword is hit in the paragraph through a logical "OR" operation. The Boolean matching algorithm is executed based on the KMP string matching algorithm or the Aho-Corasick multi-pattern matching algorithm. Each paragraph is matched with the field template one by one to generate field keyword hit data, which includes the paragraph number, the matching field name, and the hit keyword. Then, a keyword hit position extraction operation is performed on the field keyword hit data. The relative position (relative to the paragraph start character index) of each hit keyword is recorded in the paragraph text to construct field position candidate data in the format of field name, paragraph number, keyword index position, keyword length, etc., for accurately positioning the text area where the field is located. After obtaining the field position candidate data, a neighboring context extraction operation is performed on each hit position. A certain length of text range (usually 30-60 characters) is extracted forward and backward to capture the local phrase containing the field value. For example, if the hit keyword is "contract number", the substring "contract number is ABC-2024-XY45" is extracted as the original field segment. This step combines grammar heuristic rules, such as identifying the area where the field value appears after conjunctions such as "is", "are", ":", etc., to trim out the accurate original field segment data. Finally, noise character removal processing is performed on the original field segment data. This step uses regular cleaning rules to remove invalid characters, such as repeated punctuation marks (such as "::", ".."), illegal line breaks, and garbled characters (such as ) etc.; and normalize the spaces, quotation marks, etc. around the field value. The field value after removal is the field candidate content data, which includes information such as the cleaned field value text, the corresponding field name, the paragraph number to which it belongs, and the start and end indexes of the original segment, laying a foundation for subsequent field content recognition and structured output.

[0148] Therefore, the embodiments should be considered as exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of the equivalents of the application are intended to be included within the invention.

[0149] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features of the invention herein.

Claims

1. A method for extracting information from electronic archives, characterized in that, Includes the following steps: Step S1: Obtain the electronic document to be processed and perform OCR recognition to generate initial text data; Perform error detection on the initial text data to identify potential OCR misrecognitions; Step S2: For each OCR misidentification candidate, generate a first data name based on its contextual semantics and layout structure; extract the low-level features of the first data name, and use a preset domain vocabulary in combination with the first data name to perform named entity recognition on each OCR misidentification candidate, thereby assigning a second data name. Step S3: Utilize the dual mapping relationship between the first data naming and the second data naming to automatically correct and re-identify the OCR misidentified candidates, generating corrected and accurate text data; Step S4: Extract key information from the corrected and accurate text data according to the preset information field template, and output structured information.

2. The method for extracting information from electronic archives according to claim 1, characterized in that, Step S1 includes the following steps: Step S11: Obtain the electronic file to be processed; Step S12: Perform image analysis and region division on the electronic archives to be processed, extract structured text image regions, and obtain image partition data; Step S13: Perform OCR recognition on the image partition data to extract the text content in the electronic document and generate initial text data; perform character confidence analysis on the initial text data to extract the positions of characters with recognition confidence below a preset threshold and generate low-confidence character annotation data; Step S14: Combine the initial text data with the low-confidence character annotation data, perform semantic integrity analysis, identify segments with semantic missing or logical interruptions, and generate semantic anomaly candidate segment data; perform intersection location on the low-confidence character annotation data and the semantic anomaly candidate segment data, annotate the OCR misidentification area, and generate OCR misidentification candidate data.

3. The method for extracting information from electronic archives according to claim 2, characterized in that, In step S13, OCR recognition is performed on the image partition data to extract the text content from the electronic document, including: The image partitioning data is divided into image partitioning data blocks with a unit size of 128×128 pixels. Edge detection and connected component analysis are performed on each partition, and connected components with an area of ​​not less than 200 pixels are selected. 2 Suspected text areas with aspect ratios between 2:1 and 10:1; Perform character-level OCR recognition in each suspected area. It supports recognizing characters with sizes ranging from 12pt to 36pt. For characters smaller than 10pt, double pixel interpolation is performed before recognition to obtain the recognized character information. The identified character information is evaluated for confidence and marked as anomalies. Characters with a confidence level of less than 85% are marked as low-confidence characters. When the average confidence level of the entire text is higher than 92% and the recognition error rate is lower than 3%, it is marked as a valid text segment. Based on the valid text segments, the original coordinates of the electronic file to be processed are mapped and the position is reconstructed. The images are then stitched and sorted according to the order from top to bottom and from left to right, and the output is structured initial text data with field numbers, character content, position information, character spacing and average confidence.

4. The method for extracting information from electronic archives according to claim 1, characterized in that, In step S2, for each OCR misidentification candidate, a first data name is generated based on its contextual semantics and layout structure, including: For each OCR misidentified candidate, extract the corresponding preceding and following text lines to generate context text segment data; Perform part-of-speech tagging and syntactic structure analysis on the context text segment data, extract sentence structure and keyword sequences, and generate semantic analysis result data; By matching the OCR misidentification candidate data with the semantic analysis results data, and matching the field names or common phrases in the standard terminology database, semantic matching candidate field data is generated. Extract the relative position, alignment mode, and font style of the OCR misidentified candidate data in the image partition data to generate page layout feature data; The system integrates semantic matching candidate field data with layout structure feature data, performs context consistency scoring, and generates the first data name based on the highest score result.

5. The method for extracting information from electronic archives according to claim 1, characterized in that, Step S2 extracts low-level features from the first data naming and uses a preset domain vocabulary in conjunction with the first data naming to perform named entity recognition on each OCR misidentification candidate, including: The first data name is subjected to character-level encoding and sub-word segmentation to extract its low-level language features such as character length, stem structure and naming format, and generate naming low-level feature data; The system uses named low-level feature data to match pre-defined domain terms and generates candidate domain entity label data based on rules. The first data name and candidate domain entity label data are used to perform named entity classification scoring to identify misclassified or ambiguous entity ranges and generate entity confidence score data. Based on the entity confidence score data, the final entity type is determined for each OCR misidentification candidate, and named entity label binding is performed to generate a second data name.

6. The method for extracting information from electronic archives according to claim 1, characterized in that, The method for confirming the dual mapping relationship between the first data naming and the second data naming in step S3 includes: The first and second data names are compared by position index, and the relative position mapping information of the corresponding items in the original text is extracted to generate named position mapping data. Based on the naming position mapping data, a similarity measurement is performed on the character content of the first data name and the second data name to generate naming semantic similarity data; Based on the naming semantic similarity data, name pairs with similarity higher than a preset threshold are selected, an initial bidirectional naming mapping table is constructed, and a candidate dataset for naming mapping is generated. Conflict checking and ambiguity alignment are performed on the candidate datasets for naming mapping, and ambiguous naming bindings are filtered out, thereby obtaining a dual mapping relationship between the first and second data names.

7. The method for extracting information from electronic archives according to claim 1, characterized in that, Step S3 utilizes the dual mapping relationship between the first and second data naming to automatically correct and re-identify OCR misidentified candidates, including: Using a dual mapping relationship, semantic replacement matching of incorrectly identified OCR candidate options is performed, successfully matched candidate options and their suggested replacements are extracted, and error correction matching data is generated. Based on the error correction matching data, target character replacement is performed at the corresponding position in the electronic file to be processed to generate preliminary corrected text data; Contextual language validation is performed on the initially corrected text data to determine semantic coherence, resulting in incoherent segments. These incoherent segments are then rewritten using probability-optimal methods to generate context-consistent corrected data. The initial corrected text is updated based on the context consistency correction data to form the corrected and accurate text data.

8. The method for extracting information from electronic archives according to claim 7, characterized in that, Performing contextual language validation on the initially revised text data to determine semantic coherence reveals incoherent segments, including: Syntactic structure analysis was performed on the preliminary revised text data to extract the grammatical dependency matrix of the text. The sentence length was set between 5 and 40 words, and the part exceeding the range was divided into multiple clauses according to the natural semantic boundaries. For each text clause, a contextual coherence score is calculated, with the coherence score ranging from 0 to 1. Bidirectional prediction probabilities are calculated for the current sentence and the sentences before and after it. When the joint prediction probability between sentences is less than 0.35, it is initially marked as a suspicious breakpoint. By introducing semantic nesting relationships and entity continuity analysis, if the subject entities of two consecutive clauses lack logical connection or there is an obvious verb-object collocation error, they are marked as paragraphs with abnormal semantic structure. Context window resampling verification is performed on paragraphs with semantic structure abnormalities based on all suspicious breakpoints. The window size is 2 sentences before and after the paragraph. If the coherence score is less than 0.4 in more than 3 out of 5 samples, the paragraph is finally confirmed as a disjoint segment.

9. The method for extracting information from electronic archives according to claim 1, characterized in that, Step S4 includes the following steps: Step S41: Segment the corrected and accurate text data, extract semantic paragraph ranges, and generate structured paragraph boundary data; Step S42: Perform template field keyword matching on each paragraph area based on the structured paragraph boundary data, extract candidate fragments of the matching fields, and generate candidate content data for the fields; Step S43: Perform semantic normalization on the candidate content data of the field, remove redundant symbols and unify the unit format to generate standardized data of the field; Step S44: Map the standardized field data to the preset information field template, extract the corresponding field values, and generate structured information field data; perform field integrity verification on the structured information field data, remove missing key fields or abnormal field values, and output structured information.

10. An information retrieval system for electronic archives, characterized in that, For performing the information extraction method for electronic archives as described in claim 1, the information extraction system for electronic archives includes: The error detection module is used to acquire the electronic document to be processed and perform OCR recognition to generate initial text data; it performs error detection on the initial text data to identify OCR error candidates. The data naming module is used to generate a first data name for each OCR misidentified candidate based on its contextual semantics and layout structure; extract the low-level features of the first data name; and use a preset domain vocabulary in combination with the first data name to perform named entity recognition on each OCR misidentified candidate, thereby assigning a second data name. The automatic correction module is used to automatically correct and re-identify OCR misidentified candidates by utilizing the dual mapping relationship between the first data naming and the second data naming, and generate corrected and accurate text data. The information extraction module is used to extract key information from corrected and accurate text data according to preset information field templates and output structured information.

Images