An archive information extraction and intelligent management system based on self-supervised learning

Abstract

The application discloses an archive information extraction and intelligent management system based on self-supervised learning, comprising the following modules: a collection preprocessing module, which is used for collecting archive original data and generating layout sample data set and original text mapping table; a self-supervised pre-training module, which is used for obtaining a three-modal unified encoder model; a document analysis module, which is used for text detection, character recognition and table structure recovery, and generates a document analysis data set; an information extraction module, which is used for pointer positioning and optimal transmission alignment, and outputs metadata and relationship data set and registers evidence items; an alignment storage module, which is used for generating a standardized record library and an evidence linking table based on a structure-enhanced double-tower vector recall; a strategy operation and maintenance module, which is used for permission control, desensitization, archiving and incremental updating. Through the combination of three-modal self-supervision and evidence chain, the application realizes accurate extraction, alignment storage and traceable management of archive elements.

Description

Technical Field

[0001] This invention relates to the field of information processing and archival informatization technology, and in particular to an archival information extraction and intelligent management system based on self-supervised learning. Background Technology

[0002] With the rapid growth in the scale of electronic archives and the increasing online nature of business operations, the need for automatic identification, key information extraction, and end-to-end management of archive content from multiple sources and across different formats is becoming increasingly prominent. Existing technologies are mostly based on a pipeline of "optical recognition + layout analysis + rule configuration" or customized training on a small number of templates, but they generally suffer from the following problems in complex real-world environments:

[0003] Inconsistent scanning quality, skew distortion, seal obstruction, and handwritten annotations lead to instability in structural boundaries and text positioning, making errors easily amplified in subsequent stages. Information extraction and template adaptation typically require extensive manual annotation and frequent rule maintenance, resulting in high costs and slow deployment when migrating to new invoices, official documents, or business forms. Numerous heterogeneous spellings of dates, document numbers, units, and organization names make it difficult for traditional rules or dictionary matching to maintain global consistency and deduplication / disambiguation across pages, documents, and batches. Retrieval often remains at the text level or single-vector recall, failing to pinpoint the exact location and layout coordinates of the original text, disrupting the review, evidence collection, and verification process. Models are mostly updated offline, making it difficult to distill first-line proofreading feedback into controllable incremental improvements; cross-unit data sharing is limited, and privacy and continuous optimization are often difficult to simultaneously satisfy.

[0004] Therefore, how to provide a self-supervised learning-based archive information extraction and intelligent management system is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] One objective of this invention is to propose a self-supervised learning-based archive information extraction and intelligent management system. Compared with existing processes that rely on high annotation and rule stacking, this invention has the advantages of strong robustness to low annotation and complex layouts, high extraction and alignment accuracy, traceable and auditable evidence, high retrieval and recall efficiency, and incremental updates.

[0006] According to an embodiment of the present invention, an archival information extraction and intelligent management system based on self-supervised learning includes the following modules:

[0007] The data acquisition and preprocessing module is used to acquire the original data set of the archives, perform geometric regression correction, homomorphic denoising and self-attention page segmentation, generate page sample datasets, and establish the original text mapping table;

[0008] The self-supervised pre-training module is used to perform mask layout reconstruction and cross-view comparison self-supervised training on the layout sample dataset to obtain a three-modal unified encoder model.

[0009] The document parsing module is used to perform differentiable boundary optimized text detection, character recognition, table structure recovery and seal handwriting parsing based on the trimodal unified encoder model, generate a document parsing dataset, and associate it with the original text mapping table;

[0010] The information extraction module is used to call the information extraction model to perform pointer network span positioning, conditional random field sequence labeling and optimal transmission field alignment, output metadata and relation datasets, and register evidence entry indexes;

[0011] The alignment and storage module is used to read metadata, relational datasets, and knowledge graph libraries, build a structure-enhanced dual-tower vector recall model, establish a small world graph index and a product quantization index, and generate a standardized record library and evidence link table.

[0012] The strategy operation and maintenance module is used to implement access control, desensitization, automatic archiving, borrowing approval and due date reminders based on a standardized record library, evidence link table and original text mapping table. It also records verification and generates pseudo-tags for periodic incremental updates.

[0013] A method for extracting and intelligently managing archival information based on self-supervised learning according to an embodiment of the present invention includes the following steps:

[0014] The original dataset of the archives was collected, and geometric regression correction, homomorphic denoising and self-attention page segmentation were used to perform skew correction and segmentation to obtain the page sample dataset and generate the original text mapping table.

[0015] Using the layout sample dataset as input, a three-modal unified encoder model is obtained by performing representation learning through self-supervised joint pre-training of mask layout reconstruction and cross-view comparison.

[0016] The layout sample dataset is processed based on a three-modal unified encoder model. Cascaded detection and recognition, graph neural network table parsing are used for localization and reconstruction to obtain a document parsing dataset. The original text mapping table is then used for backtracking.

[0017] Taking the document parsing dataset as input, the information extraction model is invoked, and the pointer network and optimal transmission are used for joint decoding to perform entity localization and key-value alignment, thereby obtaining metadata and relation datasets.

[0018] Using metadata, relational datasets, and knowledge graph libraries as input, a structure-enhanced dual-tower vector recall model is constructed. Global vectors and field anchor vectors are used, with inner product and anchor matching as the metric. Small world graph index and product quantization index are combined to solve for alignment, disambiguation, and consistency, resulting in a standardized record library and evidence link table.

[0019] Based on a standardized record library, and using an evidence linking table and a policy rule engine, the system implements access control, desensitization, automatic archiving, borrowing approval and due date reminders, and records are verified and pseudo-labels are generated for feedback.

[0020] Optionally, the original dataset of the collected archives is subjected to geometric regression correction, homomorphic denoising, and self-attention page segmentation to perform skew correction and segmentation, resulting in a page sample dataset, and generating a text mapping table, specifically including:

[0021] Collect scanned copies of paper documents, electronic documents, and structured forms, unify them into a parsable format, and record the source number and timestamp to form the original dataset of the archives;

[0022] Geometric regression correction is performed on the page image, and homomorphic denoising and adaptive binarization are applied in the logarithmic domain to obtain the corrected page image and correction parameter log.

[0023] A self-attention page segmentation network is constructed. Line segment candidates are generated by combining the corrected page image with connected components and vertical projection. The text prior is formed by overlaying the OCR character box rasterization. The input consists of the corrected page image, text prior, 2D relative coordinate embedding and hierarchical path embedding. The network adopts a hierarchical self-attention encoder and a multi-scale fusion neck. Multi-head decoding is set for block category, boundary vertex regression, hierarchical label and reading order score. The training objective is the weighted sum of classification cross-entropy, boundary IoU or DIoU, hierarchical sequence cross-entropy and order consistency loss. Data augmentation with coordinate perturbation and random occlusion is applied.

[0024] The self-attention page segmentation network outputs block categories, boundary polygons, hierarchical labels, and reading order scores. It then performs page-block segmentation to generate a page sample dataset containing page numbers, block identifiers, page coordinates, hierarchical paths, reading order tags, and content hashes.

[0025] A source mapping table is established, with fields including source number, file path, page number, block identifier, page coordinates, hierarchical path, reading order marker, content hash, and correction parameter log index. Based on the source mapping table, a one-to-many binding is established between the original archive dataset, the corrected page diagram, and the layout sample dataset.

[0026] Optionally, the step of using a layout sample dataset as input and performing self-supervised joint pre-training of mask layout reconstruction and cross-view comparison to learn representations and obtain a three-modal unified encoder model specifically includes:

[0027] The page sample dataset is aggregated with page and block identifiers according to the original text mapping table to form training samples of text sequence, page image slice, two-dimensional relative coordinates and hierarchical path. The reading order label is retained as a regularization constraint, and the same page three modalities are positive pairs, and cross-page or cross-file pairs are negative pairs.

[0028] A pre-trained backbone for a trimodal unified encoder model is constructed, comprising a text sub-encoder, a visual sub-encoder, and a coordinate embedding module. The three features are aligned and aggregated in the fusion layer, and the output is a page-level vector and a block-level vector in the same representation space.

[0029] Two types of self-supervised tasks are set up: mask layout reconstruction and cross-view comparison alignment. The categories, boundaries and contents are reconstructed for the masked page blocks and characters, the positive and negative pairs are compared and learned, and the reading order consistency constraint is applied.

[0030] We optimize the loss by weighted sum of reconstruction loss and contrast loss, and combine in-batch negative sampling and hard negative sampling. During training, we apply Gaussian perturbation to page coordinates and perform random occlusion enhancement on page blocks, while keeping the page and block identifiers in the original text mapping table unchanged.

[0031] After completing the joint pre-training, the pre-training parameters of the trimodal unified encoder model are generated and the model version is registered.

[0032] Optionally, the step of processing the layout sample dataset based on the three-modal unified encoder model, using cascaded detection and recognition, graph neural network table parsing, for localization and reconstruction to obtain the document parsing dataset, and backtracking according to the original text mapping table, specifically includes:

[0033] The page layout sample dataset is loaded with page and block identifiers according to the original text mapping table. The affine and perspective parameters of the correction parameter log are read to complete the page correction. The reading order mark and two-dimensional relative coordinates are combined, and the three-modal unified encoder model is called to generate page-level and block-level features while maintaining the consistency of the identifiers.

[0034] A text detection method with differentiable boundary optimization is adopted to simultaneously predict the text probability map and threshold map. A soft binary map is obtained by using a differentiable threshold approximation function. Connectivity extraction, centroid offset correction, and polygon fitting are performed. Boundary polygons and block confidence are registered and associated with block identifiers.

[0035] Candidate text blocks are cropped according to the log inverse transform of the correction parameters and normalized with a fixed height. Homomorphic denoising and adaptive binarization are performed. Convolutional features are extracted and unfolded into a temporal sequence along the reading direction. Character probabilities are calculated and decoded using a connection-time classification bundle search. In the bundle search stage, the scores are re-scored and format constraints are applied based on a restricted dictionary and regular template. The output is a character sequence, character confidence, and word confidence. Character coordinates and word coordinates are generated by back-projection of temporal position and boundary polygon.

[0036] Connectivity detection and line segment detection are used to generate candidate cell boundaries for the candidate table region. A relationship graph is constructed with cells as nodes and adjacency and alignment relationships as edges. Under row and column uniqueness constraints and merging consistency constraints, row and column ownership and merging relationships are jointly solved. Cross-page headers are inherited with structural consistency constraints. A table structure graph and the mapping of cells to page coordinates and text are generated.

[0037] Pixel-level masks and outlines are obtained by instance region segmentation for the stamp area and handwritten area. Learnable sharpening items are set at the boundaries. Text sequence decoding is used to parse the annotation content. The results are registered as independent blocks and the page identifier, block identifier, page coordinates and hierarchical path are kept consistent.

[0038] The boundary polygon, text baseline, and cell vertex coordinates are inversely transformed to the page coordinate system according to the calibration parameter log. The block-level confidence and cell-level confidence are calculated by integrating geometric consistency, character confidence, and word confidence. The hierarchical path and reading order score are generated based on proximity, inclusion relationship, and reading order constraints. The page-level and block-level results are summarized to form a document parsing dataset, and the backtracking relationship is established using the original text mapping table.

[0039] Optionally, the step of taking the document parsing dataset as input, calling the information extraction model, using pointer networks and optimal transmission for joint decoding, performing entity localization and key-value alignment, and obtaining metadata and relation datasets specifically includes:

[0040] The document parsing dataset and the original text mapping table are aggregated according to page identifier and block identifier, including text within the block, page coordinates, block type, table structure diagram and reading order label, and field dictionary, format template and required constraints are generated from the pre-set knowledge graph library as extraction priors;

[0041] An information extraction model is constructed, the encoder is initialized with a three-modal unified encoder model, the backbone parameters are frozen and a low-rank adaptation layer is superimposed to obtain block-level representation and context window representation;

[0042] The starting point probability and ending point probability are calculated using a pointer network. A set of candidate spans is generated based on threshold, span length and block boundary constraints. Evidence page coordinates and block identifiers are registered for each candidate span.

[0043] Conditional random fields are used for sequence labeling, and BIO tag sequences are output. Consistency constraints are applied to the BIO tag sequences and candidate span sets to generate entity candidate sets. The entity sets and entity categories are determined according to conflict resolution and format template constraints, and the source candidate span identifier and page coordinates are retained for each entity.

[0044] Based on the field dictionary, a field anchor point representation is generated for each target field. Candidate value representations are extracted from the entity set and table structure diagram. The corresponding double affine score is calculated to form a cost matrix. Under the constraints of mandatory field, uniqueness, and format template, the field value mapping and pairing confidence are obtained by optimal transmission decoding.

[0045] An entity relationship graph is constructed based on the entity set. Adjacency within the same paragraph and adjacency across blocks are set. Message passing and joint classification are performed to obtain entity relationship triples. Time standardization, unit conversion and document number structure parsing are performed. Cross-block deduplication and consistency verification are performed according to reading order and proximity relationship to form metadata and relationship dataset.

[0046] Optionally, the constructed structure-enhanced dual-tower vector recall model uses global vectors and field anchor vectors, with inner product and anchor matching as the metric. It combines small-world graph indexes and product quantization indexes to perform alignment, disambiguation, and consistency calculations, resulting in a standardized record library and evidence linking table. Specifically, this includes:

[0047] Read metadata and relational datasets, knowledge graph library and original text mapping table, load field anchor representations, candidate value representations, entity sets and cost matrices, and associate evidence entry indexes with page identifiers and block identifiers;

[0048] A structure-enhanced dual-tower vector recall model is constructed. Both the query encoder and the candidate encoder are self-attention-based transformer encoders with non-shared parameters. The input consists of text tags, two-dimensional relative coordinates, and block type embeddings. The sequence representation is obtained through multi-head attention and feedforward network. Then, a global vector is generated by weighted common tag pooling. At the same time, a field anchor sequence is generated using field name or field template as seed and the field anchor vector is obtained through the same path. All vectors are processed by layer normalization and norm normalization while maintaining page coordinates and block identifier references.

[0049] Define a weighted matching score, which is the weighted sum of the global vector inner product score and the optimal transmission matching score of the field anchor point as the pairing score. The weight coefficients are denoted as the first coefficient and the second coefficient.

[0050] Establish a vector retrieval index, construct a hierarchical navigable small world graph index for global vectors, construct a product quantization index for field anchor vectors, register the index handle and establish a reference with the page identifier and block identifier in the original text mapping table;

[0051] Entity and field candidates are retrieved from the small-world graph index and the product quantization index based on inner product similarity. A weighted matching score is calculated by combining the cost matrix, format template and mandatory constraints to generate a candidate pairing set.

[0052] The field consistency constraint, time sequence consistency constraint, and document number structure constraint are modeled as integer linear programming. The candidate pairing set is solved to obtain entity alignment results, same name disambiguation results, and field value mapping, and the solution log and unsatisfied constraint list are output.

[0053] The global vector is optimized by contrastive learning loss, and the transmission cost between field anchors and candidate values ​​is minimized by anchor matching loss. For hard examples, hard example sampling and temperature scaling calibration based on pairwise scores are used to learn the first and second coefficients and keep them consistent with the cost matrix.

[0054] Time is standardized according to a time template, quantity and unit are standardized according to a unit conversion table, institution and region are standardized according to a coding table, and document number is parsed and its constituent elements are recorded according to a structural template, generating standardized field values ​​and writing them into a standardized record library, while also generating an evidence link table.

[0055] Optionally, the system, based on a standardized record library and an evidence linking table and policy rule engine, implements access control, data anonymization, automatic archiving, borrowing approval, and due date reminders. It also verifies records and generates pseudo-tags for feedback. Specifically, this includes:

[0056] Read the standardized record library, evidence link table, original text mapping table and index handle, and obtain the page coordinates, block identifiers and character coordinates according to the record identifier;

[0057] The policy rule engine performs access control, desensitization, and process processing. Visible fields are clipped based on RBAC and access control tables. Desensitization is implemented according to regular expression templates, sensitive word libraries, and coordinate masking. Automatic archiving and borrowing approval are completed by a directed state machine. Expiration reminders are generated based on time templates and timers.

[0058] Based on the evidence link table, return the record identifier, page coordinates, block identifier, and character coordinates, and output the source field;

[0059] Record manual proofreading, generate pseudo-label sample set, filter field values ​​and relationships using consistency rules, format templates and thresholds, and register evidence entry index through conflict filtering and deduplication;

[0060] The parameters of the low-rank adaptation layer of the trimodal unified encoder model, information extraction model and structure-enhanced dual-tower vector recall model are adjusted, and federated average aggregation and differential privacy noise injection are adopted, and the privacy budget and model version are recorded.

[0061] Write operations to the operation log and audit trail, and associate new records, pseudo-label samples, model version numbers, task results and timestamps with the original text mapping table.

[0062] The beneficial effects of this invention are:

[0063] Compared to existing "OCR plus rule" pipelines, this invention offers the following advantages: It significantly improves robustness to complex layouts such as tilt, perspective distortion, stamp occlusion, and handwritten annotations by learning robust representations of layout, text, and coordinates under low-annotation conditions through a trimodal self-supervised unified encoder; it reduces error propagation caused by block boundaries and text line breaks by employing differentiable boundary-optimized layout parsing and connection-time classification; it reduces mis-extraction and mismatching while also considering table structure backfilling by using pointer networks and conditional random field consistency constraints for optimal transmission of field alignment; it introduces structure-enhanced dual-tower vector recall and consistency solving to achieve globally optimal decisions for entity alignment and name disambiguation; it establishes a bidirectional traceable link from records to original text coordinates through an evidence link table, facilitating verification and auditing; it combines a policy rule engine with a standardized record library to complete access control, desensitization, and automatic archiving, forming a closed-loop iteration with pseudo-label reflow and efficient parameter fine-tuning, continuously improving accuracy and reducing rule maintenance and manual annotation costs; and it employs federated aggregation and differential privacy to balance cross-unit data compliance and continuous optimization. Attached Figure Description

[0064] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0065] Figure 1 This is a schematic diagram of the structure of a self-supervised learning-based intelligent management system for extracting and managing archival information proposed in this invention.

[0066] Figure 2 This is a flowchart of a self-supervised learning-based method for extracting and intelligently managing archival information, as proposed in this invention.

[0067] Figure 3 This is a framework diagram of the structure-enhanced dual-tower vector recall model in a self-supervised learning-based method for extracting and intelligently managing archival information proposed in this invention. Detailed Implementation

[0068] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0069] refer to Figure 1 A self-supervised learning-based intelligent management system for extracting and managing archival information includes the following modules:

[0070] The data acquisition and preprocessing module is used to acquire the original data set of the archives, perform geometric regression correction, homomorphic denoising and self-attention page segmentation, generate page sample datasets, and establish the original text mapping table;

[0071] The self-supervised pre-training module is used to perform mask layout reconstruction and cross-view comparison self-supervised training on the layout sample dataset to obtain a three-modal unified encoder model.

[0072] The document parsing module is used to perform differentiable boundary optimized text detection, character recognition, table structure recovery and seal handwriting parsing based on the trimodal unified encoder model, generate a document parsing dataset, and associate it with the original text mapping table;

[0073] The information extraction module is used to call the information extraction model to perform pointer network span positioning, conditional random field sequence labeling and optimal transmission field alignment, output metadata and relation datasets, and register evidence entry indexes;

[0074] The alignment and storage module is used to read metadata, relational datasets, and knowledge graph libraries, build a structure-enhanced dual-tower vector recall model, establish a small world graph index and a product quantization index, and generate a standardized record library and evidence link table.

[0075] The strategy operation and maintenance module is used to implement access control, desensitization, automatic archiving, borrowing approval and due date reminders based on a standardized record library, evidence link table and original text mapping table. It also records verification and generates pseudo-tags for periodic incremental updates.

[0076] refer to Figure 2-3 A method for extracting and intelligently managing archival information based on self-supervised learning includes the following steps:

[0077] The original dataset of the archives was collected, and geometric regression correction, homomorphic denoising and self-attention page segmentation were used to perform skew correction and segmentation to obtain the page sample dataset and generate the original text mapping table.

[0078] Using the layout sample dataset as input, a three-modal unified encoder model is obtained by performing representation learning through self-supervised joint pre-training of mask layout reconstruction and cross-view comparison.

[0079] The layout sample dataset is processed based on a three-modal unified encoder model. Cascaded detection and recognition, graph neural network table parsing are used for localization and reconstruction to obtain a document parsing dataset. The original text mapping table is then used for backtracking.

[0080] Taking the document parsing dataset as input, the information extraction model is invoked, and the pointer network and optimal transmission are used for joint decoding to perform entity localization and key-value alignment, thereby obtaining metadata and relation datasets.

[0081] Using metadata, relational datasets, and knowledge graph libraries as input, a structure-enhanced dual-tower vector recall model is constructed. Global vectors and field anchor vectors are used, with inner product and anchor matching as the metric. Small world graph index and product quantization index are combined to solve for alignment, disambiguation, and consistency, resulting in a standardized record library and evidence link table.

[0082] Based on a standardized record library, and using an evidence linking table and a policy rule engine, the system implements access control, desensitization, automatic archiving, borrowing approval and due date reminders, and records are verified and pseudo-labels are generated for feedback.

[0083] In this embodiment, the original dataset of the collected archives is subjected to geometric regression correction, homomorphic denoising, and self-attention page segmentation to perform skew correction and segmentation, resulting in a page sample dataset, and generating a text mapping table, specifically including:

[0084] Collect scanned copies of paper documents, electronic documents, and structured forms, unify them into a parsable format, and record the source number and timestamp to form the original dataset of the archives;

[0085] Geometric regression correction is performed on the page image, and homomorphic denoising and adaptive binarization are applied in the logarithmic domain to obtain the corrected page image and correction parameter log.

[0086] A self-attention page segmentation network is constructed. Line segment candidates are generated by combining the corrected page image with connected components and vertical projection. The text prior is formed by overlaying the OCR character box rasterization. The input consists of the corrected page image, text prior, 2D relative coordinate embedding and hierarchical path embedding. The network adopts a hierarchical self-attention encoder and a multi-scale fusion neck. Multi-head decoding is set for block category, boundary vertex regression, hierarchical label and reading order score. The training objective is the weighted sum of classification cross-entropy, boundary IoU or DIoU, hierarchical sequence cross-entropy and order consistency loss. Data augmentation with coordinate perturbation and random occlusion is applied.

[0087] The self-attention page segmentation network outputs block categories, boundary polygons, hierarchical labels, and reading order scores. It then performs page-block segmentation to generate a page sample dataset containing page numbers, block identifiers, page coordinates, hierarchical paths, reading order tags, and content hashes.

[0088] A source mapping table is established, with fields including source number, file path, page number, block identifier, page coordinates, hierarchical path, reading order marker, content hash, and correction parameter log index. Based on the source mapping table, a one-to-many binding is established between the original archive dataset, the corrected page diagram, and the layout sample dataset.

[0089] This implementation generates a high-quality page layout sample dataset through geometric correction, homomorphic denoising, and self-attention page segmentation, and solidifies the relationship between pages, blocks, and the original text with a mapping table, which significantly improves the stability of subsequent recognition and extraction and the availability of training samples.

[0090] In this embodiment, the step of using a layout sample dataset as input, employing self-supervised joint pre-training of mask layout reconstruction and cross-view comparison, and performing representation learning to obtain a three-modal unified encoder model specifically includes:

[0091] The page sample dataset is aggregated with page and block identifiers according to the original text mapping table to form training samples of text sequence, page image slice, two-dimensional relative coordinates and hierarchical path. The reading order label is retained as a regularization constraint, and the same page three modalities are positive pairs, and cross-page or cross-file pairs are negative pairs.

[0092] A pre-trained backbone for a trimodal unified encoder model is constructed, comprising a text sub-encoder, a visual sub-encoder, and a coordinate embedding module. The three features are aligned and aggregated in the fusion layer, and the output is a page-level vector and a block-level vector in the same representation space.

[0093] Two types of self-supervised tasks are set up: mask layout reconstruction and cross-view comparison alignment. The categories, boundaries and contents are reconstructed for the masked page blocks and characters, the positive and negative pairs are compared and learned, and the reading order consistency constraint is applied.

[0094] We optimize the loss by weighted sum of reconstruction loss and contrast loss, and combine in-batch negative sampling and hard negative sampling. During training, we apply Gaussian perturbation to page coordinates and perform random occlusion enhancement on page blocks, while keeping the page and block identifiers in the original text mapping table unchanged.

[0095] After completing the joint pre-training, the pre-training parameters of the trimodal unified encoder model are generated and the model version is registered.

[0096] This implementation method learns a unified representation of layout, text, and coordinates under unlabeled or poorly labeled conditions through three-modal self-supervised pre-training of mask layout reconstruction and cross-view comparison, thereby enhancing robustness across templates and low-quality scans and reducing manual annotation and migration costs.

[0097] In this embodiment, the process of processing the layout sample dataset based on the three-modal unified encoder model, using cascaded detection and recognition, graph neural network table parsing, for localization and reconstruction, to obtain the document parsing dataset, and backtracking according to the original text mapping table, specifically includes:

[0098] The page layout sample dataset is loaded with page and block identifiers according to the original text mapping table. The affine and perspective parameters of the correction parameter log are read to complete the page correction. The reading order mark and two-dimensional relative coordinates are combined, and the three-modal unified encoder model is called to generate page-level and block-level features while maintaining the consistency of the identifiers.

[0099] A text detection method with differentiable boundary optimization is adopted to simultaneously predict the text probability map and threshold map. A soft binary map is obtained by using a differentiable threshold approximation function. Connectivity extraction, centroid offset correction, and polygon fitting are performed. Boundary polygons and block confidence are registered and associated with block identifiers.

[0100] Candidate text blocks are cropped according to the log inverse transform of the correction parameters and normalized with a fixed height. Homomorphic denoising and adaptive binarization are performed. Convolutional features are extracted and unfolded into a temporal sequence along the reading direction. Character probabilities are calculated and decoded using a connection-time classification bundle search. In the bundle search stage, the scores are re-scored and format constraints are applied based on a restricted dictionary and regular template. The output is a character sequence, character confidence, and word confidence. Character coordinates and word coordinates are generated by back-projection of temporal position and boundary polygon.

[0101] Connectivity detection and line segment detection are used to generate candidate cell boundaries for the candidate table region. A relationship graph is constructed with cells as nodes and adjacency and alignment relationships as edges. Under row and column uniqueness constraints and merging consistency constraints, row and column ownership and merging relationships are jointly solved. Cross-page headers are inherited with structural consistency constraints. A table structure graph and the mapping of cells to page coordinates and text are generated.

[0102] Pixel-level masks and outlines are obtained by instance region segmentation for the stamp area and handwritten area. Learnable sharpening items are set at the boundaries. Text sequence decoding is used to parse the annotation content. The results are registered as independent blocks and the page identifier, block identifier, page coordinates and hierarchical path are kept consistent.

[0103] The boundary polygon, text baseline, and cell vertex coordinates are inversely transformed to the page coordinate system according to the calibration parameter log. The block-level confidence and cell-level confidence are calculated by integrating geometric consistency, character confidence, and word confidence. The hierarchical path and reading order score are generated based on proximity, inclusion relationship, and reading order constraints. The page-level and block-level results are summarized to form a document parsing dataset, and the backtracking relationship is established using the original text mapping table.

[0104] This implementation method accurately recovers block boundaries, character and table structures through differentiable boundary detection, connection time sequence classification and recognition, and relationship graph table parsing. It outputs character and word coordinates and confidence scores, reduces line breaks, adhesion and misalignment, and improves parsing accuracy and evidence location capabilities.

[0105] In this embodiment, the step of taking the document parsing dataset as input, calling the information extraction model, using pointer networks and optimal transmission for joint decoding, performing entity localization and key-value alignment, and obtaining metadata and relation datasets specifically includes:

[0106] The document parsing dataset and the original text mapping table are aggregated according to page identifier and block identifier, including text within the block, page coordinates, block type, table structure diagram and reading order label, and field dictionary, format template and required constraints are generated from the pre-set knowledge graph library as extraction priors;

[0107] An information extraction model is constructed, the encoder is initialized with a three-modal unified encoder model, the backbone parameters are frozen and a low-rank adaptation layer is superimposed to obtain block-level representation and context window representation;

[0108] The starting point probability and ending point probability are calculated using a pointer network. A set of candidate spans is generated based on threshold, span length and block boundary constraints. Evidence page coordinates and block identifiers are registered for each candidate span.

[0109] Conditional random fields are used for sequence labeling, and BIO tag sequences are output. Consistency constraints are applied to the BIO tag sequences and candidate span sets to generate entity candidate sets. The entity sets and entity categories are determined according to conflict resolution and format template constraints, and the source candidate span identifier and page coordinates are retained for each entity.

[0110] Based on the field dictionary, a field anchor point representation is generated for each target field. Candidate value representations are extracted from the entity set and table structure diagram. The corresponding double affine score is calculated to form a cost matrix. Under the constraints of mandatory field, uniqueness, and format template, the field value mapping and pairing confidence are obtained by optimal transmission decoding.

[0111] An entity relationship graph is constructed based on the entity set. Adjacency within the same paragraph and adjacency across blocks are set. Message passing and joint classification are performed to obtain entity relationship triples. Time standardization, unit conversion and document number structure parsing are performed. Cross-block deduplication and consistency verification are performed according to reading order and proximity relationship to form metadata and relationship dataset.

[0112] This implementation combines pointer positioning with conditional random field consistency and optimal transmission alignment to form entity sets, field value mappings, and relation triples, reducing erroneous extraction and mispairing, supporting table backfilling and cross-block merging, and directly producing structured results that can be stored in the database.

[0113] In this embodiment, the construction of the structure-enhanced dual-tower vector recall model uses global vectors and field anchor vectors, with inner product and anchor matching as the metric. It combines small-world graph indexes and product quantization indexes to perform alignment, disambiguation, and consistency calculations, resulting in a standardized record library and evidence linking table. Specifically, this includes:

[0114] Read metadata and relational datasets, knowledge graph library and original text mapping table, load field anchor representations, candidate value representations, entity sets and cost matrices, and associate evidence entry indexes with page identifiers and block identifiers;

[0115] A structure-enhanced dual-tower vector recall model is constructed. Both the query encoder and the candidate encoder are self-attention-based transformer encoders with non-shared parameters. The input consists of text tags, two-dimensional relative coordinates, and block type embeddings. The sequence representation is obtained through multi-head attention and feedforward network. Then, a global vector is generated by weighted common tag pooling. At the same time, a field anchor sequence is generated using field name or field template as seed and the field anchor vector is obtained through the same path. All vectors are processed by layer normalization and norm normalization while maintaining page coordinates and block identifier references.

[0116] Define a weighted matching score, which is the weighted sum of the global vector inner product score and the optimal transmission matching score of the field anchor point as the pairing score. The weight coefficients are denoted as the first coefficient and the second coefficient.

[0117] Establish a vector retrieval index, construct a hierarchical navigable small world graph index for global vectors, construct a product quantization index for field anchor vectors, register the index handle and establish a reference with the page identifier and block identifier in the original text mapping table;

[0118] Entity and field candidates are retrieved from the small-world graph index and the product quantization index based on inner product similarity. A weighted matching score is calculated by combining the cost matrix, format template and mandatory constraints to generate a candidate pairing set.

[0119] The field consistency constraint, time sequence consistency constraint, and document number structure constraint are modeled as integer linear programming. The candidate pairing set is solved to obtain entity alignment results, same name disambiguation results, and field value mapping, and the solution log and unsatisfied constraint list are output.

[0120] The global vector is optimized by contrastive learning loss, and the transmission cost between field anchors and candidate values ​​is minimized by anchor matching loss. For hard examples, hard example sampling and temperature scaling calibration based on pairwise scores are used to learn the first and second coefficients and keep them consistent with the cost matrix.

[0121] Time is standardized according to a time template, quantity and unit are standardized according to a unit conversion table, institution and region are standardized according to a coding table, and document number is parsed and its constituent elements are recorded according to a structural template, generating standardized field values ​​and writing them into a standardized record library, while also generating an evidence link table.

[0122] This implementation achieves high recall and high precision alignment of entities and fields by using structurally enhanced dual-tower recall, small-world graph, product quantization index, and consistency constraint solution; at the same time, it retains evidence coordinates and index handles, supporting massive scale and audit traceability.

[0123] In this embodiment, the implementation of access control, data anonymization, automatic archiving, borrowing approval, and due date reminders, based on a standardized record library, an evidence linking table, and a policy rule engine, along with record verification and pseudo-tag resubmission, specifically includes:

[0124] Read the standardized record library, evidence link table, original text mapping table and index handle, and obtain the page coordinates, block identifiers and character coordinates according to the record identifier;

[0125] The policy rule engine performs access control, desensitization, and process processing. Visible fields are clipped based on RBAC and access control tables. Desensitization is implemented according to regular expression templates, sensitive word libraries, and coordinate masking. Automatic archiving and borrowing approval are completed by a directed state machine. Expiration reminders are generated based on time templates and timers.

[0126] Based on the evidence link table, return the record identifier, page coordinates, block identifier, and character coordinates, and output the source field;

[0127] Record manual proofreading, generate pseudo-label sample set, filter field values ​​and relationships using consistency rules, format templates and thresholds, and register evidence entry index through conflict filtering and deduplication;

[0128] The parameters of the low-rank adaptation layer of the trimodal unified encoder model, information extraction model and structure-enhanced dual-tower vector recall model are adjusted, and federated average aggregation and differential privacy noise injection are adopted, and the privacy budget and model version are recorded.

[0129] Write operations to the operation log and audit trail, and associate new records, pseudo-label samples, model version numbers, task results and timestamps with the original text mapping table.

[0130] This implementation method achieves location tracking by using a policy rule engine-driven permission, de-identification, and process control, combined with an evidence link table; it also uses pseudo-label feedback and efficient parameter fine-tuning to form online incremental iteration, continuously improving accuracy and reducing operation and maintenance costs.

[0131] Example 1:

[0132] To verify the feasibility of this invention in practice, it was applied to the archive management and transformation of an e-government sharing platform. The platform covers paper scans, electronic documents, and structured forms from four commissions and bureaus: Development and Reform Commission, Finance Bureau, Human Resources and Social Security Bureau, and Market Supervision Bureau. During implementation, geometric regression correction and homomorphic denoising were first used to improve image quality. Then, self-attention layout segmentation was used to establish a text mapping table. Self-supervised pre-training of a three-modal unified encoder was completed using mask layout reconstruction and cross-view comparison. Document parsing data was generated based on differentiable boundary optimization of text detection, table graph parsing, and seal / handwriting parsing. Finally, "pointer network + CRF + optimal transmission" was used for field location and pairing, and "structure-enhanced dual-tower + HNSW / PQ + integer programming constraints" were used to complete alignment, disambiguation, and consistency solutions. All fields can be traced back to page / block / character coordinates, and the strategy engine implements desensitization, approval, and expiration reminders.

[0133] Table 1. Benchmark Evaluation Sampling Set

[0134] System / Configuration End-to-end field accuracy Table Structure F1 Character recognition CER Average approval time (hours) Manual proofreading time per thousand pages (hours) Evidence traceability Rules + General OCR Baseline 0.552 0.724 0.073 57.3 162 0.822 This invention (v1 joint pre-training) 0.742 0.842 0.038 26.1 92.4 0.942 This invention (final version: trimodal + dual tower + ILP) 0.833 0.894 0.026 8.4 58.7 0.983

[0135] As can be seen from the comparison in Table 1, this invention improves the end-to-end field accuracy from 0.552 to 0.833; reduces the average approval time to 8.4 hours; reduces proofreading time to 58.7 hours / thousand pages; reduces CER from 0.073 to 0.026, significantly stabilizes the parsing of cross-page headers and merged cells; and achieves an evidence traceability rate of 0.983, meeting the requirements for audit trailing.

[0136] Table 2 Monthly Operational Data Table

[0137] month Processing Pages System field accuracy False alarm rate underreporting rate Borrowing approval SLA (hours) Save labor hours Jun-25 421,380 0.82 0.054 0.071 48.6 6,218 Jul-25 452,910 0.85 0.051 0.067 32.4 6,984 Aug-25 468,230 0.87 0.043 0.061 18.5 7,812

[0138] Table 2 shows that with the return of false labels and incremental updates for low-rank adaptation, the accuracy rate increased from 0.82 to 0.87; the false alarm / missed alarm rate decreased simultaneously, and the approval SLA decreased from 48.6 hours to 18.5 hours; more than 20,000 hours of manpower were saved in three months (including other units), significantly reducing operation and maintenance costs while maintaining high accuracy.

[0139] To verify the value of key components, ablation experiments were conducted, and the results are shown in Table 3:

[0140] Table 3 Ablation Data of Key Components

[0141] Configuration End-to-end field accuracy Field alignment accuracy Same Name Disambiguation F1 Manual proofreading time per thousand pages (hours) Complete model (control) 0.833 0.911 0.875 58.7 Differentiable boundary removal optimization 0.789 0.881 0.847 72.3 Remove optimal transport decoding 0.755 0.856 0.831 78.1 Remove dual towers + indexes 0.743 0.867 0.816 93.6 Remove Integer Programming Consistency 0.781 0.881 0.847 71.2

[0142] As can be seen from Table 3, differentiable boundary optimization directly affects detection robustness; optimal transmission ensures correct field-level pairing; dual-tower retrieval and indexing are the foundation for recall and efficiency under large-scale candidate scenarios; integer programming transforms business constraints into globally consistent solutions, significantly suppressing mismatches that appear to be correct locally.

[0143] This invention achieves highly robust extraction and alignment, significantly improving end-to-end accuracy while simultaneously reducing approval and verification costs; evidence links enable coordinate-level traceability, meeting compliance audit requirements; through self-monitoring and lightweight incremental updates, the system maintains stable performance even with changes in business distribution, achieving "more accurate extraction, more stable alignment, auditable management, and adaptable operation and maintenance."

[0144] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. An archive information extraction and intelligent management system based on self-supervised learning, characterized in that, Comprise the following modules: The acquisition preprocessing module is used for collecting the original data set of archives, performing geometric regression correction, homomorphic denoising and self-attention layout segmentation, generating a layout sample data set, and establishing an original text mapping table; The self-supervised pre-training module is used for mask layout reconstruction and cross-view contrastive self-supervised training on the layout sample data set, and constructs a three-modal unified encoder model comprising a text sub-encoder, a visual sub-encoder and a coordinate embedding module, aligns and aggregates three features at the fusion layer, and outputs page-level vectors and block-level vectors in the same representation space; The document analysis module is used for performing differentiable boundary optimization text detection, character recognition, table structure recovery and seal handwritten body analysis based on the three-modal unified encoder model, generating a document analysis data set, and associating with the original text mapping table; The information extraction module is used for using an information extraction model comprising a pointer network span positioning, a conditional random field sequence labeling and an optimal transport field alignment based on the document analysis data set and the original text mapping table, outputting an entity set, a field value mapping and a pairing confidence and registering an evidence entry index to form metadata, and constructing an entity relationship graph based on the entity set, performing message passing and joint classification according to same paragraph adjacency and cross-block adjacency to obtain entity relationship triples to form a relationship data set; The alignment and storage module is used for reading the metadata and the relationship data set and the knowledge graph database, constructing a structure-enhanced double-tower vector recall model, the query end encoder and the candidate end encoder are both transformer encoders based on self-attention, the parameters are not shared, the input is composed of text labels, two-dimensional relative coordinates and block type embeddings, the sequence representation is obtained through multi-head attention and feedforward network, and the global vector is generated through weighted common label pooling, the field anchor sequence is generated by taking the field name or field template as a seed and the field anchor vector is obtained through the same path, all vectors are processed through layer normalization and norm normalization and the page coordinates and block identifiers are referenced, the small world graph index and the product quantization index are established, the standardized record database and the evidence link table are generated; The strategy operation and maintenance module is used for implementing permission control, desensitization, automatic archiving, borrowing approval and expiration reminder based on the standardized record database and the evidence link table and the original text mapping table, recording correction and generating pseudo-label backflow for periodic incremental update.

2. The archive information extraction and intelligent management system based on self-supervised learning according to claim 1, characterized in that, The modules are realized through the following methods: The original data set of archives is collected, geometric regression correction, homomorphic denoising and self-attention layout segmentation are used for tilt correction and segmentation, the layout sample data set is obtained, and the original text mapping table is generated; The three-modal unified encoder model is obtained through the self-supervised joint pre-training of mask layout reconstruction and cross-view contrastive representation learning using the layout sample data set as input; The document analysis data set is obtained through positioning and restoration by using cascaded detection and recognition and graph neural network table analysis based on the three-modal unified encoder model processing the layout sample data set, and the original text mapping table is traced back; The metadata and the relationship data set are obtained through entity positioning and key-value alignment by calling the information extraction model and using the pointer network and the optimal transport joint decoding using the document analysis data set as input; With metadata and relational dataset and knowledge graph library as input, a structure-enhanced double-tower vector retrieval model is constructed, global vector and field anchor vector are used, inner product and anchor matching metric are used, small-world graph index and product quantization index are combined, alignment, disambiguation and consistency solution are performed, standardized record library and evidence linking table are obtained; With standardized record library as the basis, based on evidence linking table and strategy rule engine, permission control, desensitization, automatic archiving, borrowing approval and expiration reminder are implemented, record proofreading is performed and pseudo-label backflow is generated.

3. The archive information extraction and intelligent management system based on self-supervised learning according to claim 2, characterized in that, The collected archive original dataset is used for geometric regression correction, homomorphic denoising and self-attention layout segmentation to perform tilt correction and cutting to obtain a layout sample dataset, and a text mapping table is generated, specifically including: Collect paper scans, electronic documents and structured forms, unify them into a parseable format and record the source number and timestamp to form the archive original dataset; Perform geometric regression correction on the page image, apply homomorphic denoising and adaptive binarization in the logarithmic domain to obtain corrected page images and correction parameter logs; A self-attention layout segmentation network is constructed to generate line segment candidates from corrected page images combined with connected domains and vertical projections, and to form text priors by superimposing OCR character frame rasterization. The input consists of corrected page images, text priors, two-dimensional relative coordinate embeddings, and hierarchical path embeddings. The network uses hierarchical self-attention encoders and multi-scale fusion necks, and sets multi-head decoding for block category, boundary vertex regression, hierarchical label, and reading order score. The training target is the weighted sum of classification cross-entropy, boundary IoU or DIoU, hierarchical sequence cross-entropy, and order consistency loss, and data augmentation is applied with coordinate perturbation and random masking; The self-attention layout segmentation network outputs block category, boundary polygon, hierarchical label, and reading order score, performs page-block cutting, and generates a layout sample dataset containing page number, block identifier, page coordinates, hierarchical path, reading order marker, and content hash; A text mapping table is established, including source number, file path, page number, block identifier, page coordinates, hierarchical path, reading order marker, content hash, and correction parameter log index. Based on the text mapping table, the archive original dataset, corrected page images, and layout sample dataset are bound one-to-many.

4. The archive information extraction and intelligent management system based on self-supervised learning according to claim 2, characterized in that, The layout sample dataset is processed based on a three-modal unified encoder model using cascaded detection and recognition and graph neural network table parsing to locate and restore, obtaining a document parsing dataset, and backtracking according to the text mapping table, specifically including: Load page identifier and block identifier from the layout sample dataset according to the text mapping table, read affine parameters and perspective parameters from the correction parameter log to complete page correction, combine reading order marker and two-dimensional relative coordinates, and call the three-modal unified encoder model to generate page-level features and block-level features while keeping the identifiers consistent; A differentiable boundary optimization text detection method is used to simultaneously predict text probability maps and threshold maps, use a differentiable threshold approximation function to obtain soft binary maps, perform connected component extraction, centroid offset correction, and polygon fitting, register boundary polygons and block confidence, and associate block identifiers; The candidate text block is inversely transformed, cropped and fixed height normalized according to the correction parameter log, homomorphic denoising and adaptive binarization are performed, convolution features are extracted and unfolded into a time sequence along the reading direction, character probabilities are calculated and decoded by searching a connected time sequence classification bundle, the bundle search stage is rescored and format constrained according to a restricted dictionary and a regular template, and character sequences, character confidence and word confidence are output, and character coordinates and word coordinates are generated by back projection according to time sequence positions and boundary polygons; The single cell boundary candidate is generated by adopting connected domain detection and line segment detection on the table candidate region, a relationship graph is constructed with single cells as nodes and adjacency and alignment relationships as edges, the row and column attribution and merging relationship are jointly solved under the row and column uniqueness constraint and merging consistency constraint, the cross-page table header is inherited under the structure consistency constraint, and a table structure graph and the mapping of single cells to page coordinates and text are generated; The pixel-level mask and contour are obtained by adopting instance region segmentation on the seal area and handwritten area, a learnable sharpening item is set at the boundary, the content of the annotation is parsed by adopting a character sequence decoding, and the result is registered as an independent block while keeping the page identifier, block identifier, page coordinates and hierarchical path consistent; The boundary polygon, text baseline and single cell vertex coordinates are inversely transformed to the page coordinate system according to the correction parameter log, the block-level confidence and single cell-level confidence are calculated by fusing geometric consistency and character confidence and word confidence, the hierarchical path and reading order score are generated based on the proximity, inclusion relationship and reading order constraint, the page-level and block-level results are summarized to form a document parsing dataset, and a backtracking relationship is established by using an original text mapping table.

5. The archive information extraction and intelligent management system based on self-supervised learning according to claim 2, characterized in that, The document parsing dataset is input, an information extraction model is called, a pointer network and optimal transport joint decoding are used, entity positioning and key-value alignment are performed, and a metadata and relationship dataset is obtained, specifically including: The block-in-text, page coordinates, block type, table structure graph and reading order mark are gathered from the document parsing dataset and the original text mapping table according to the page identifier and block identifier, and the field dictionary, format template and mandatory constraint are generated from the pre-stored knowledge graph library as extraction priors; An information extraction model is constructed, a three-modal unified encoder model is used to initialize the encoding end, the backbone parameters are frozen and a low-rank adaptive layer is superimposed, and a block-level representation and a context window representation are obtained; The start point probability and end point probability are calculated by using a pointer network, a candidate span set is generated according to a threshold, span length and block boundary constraint, and evidence page coordinates and block identifiers are registered for each candidate span; Sequence labeling is performed by using a conditional random field, a BIO label sequence is output, the BIO label sequence and the candidate span set are subjected to consistency constraint, an entity candidate set is generated, the entity set and entity category are determined according to conflict resolution and format template constraint, and each entity is kept with the source candidate span identifier and page coordinates; The field anchor point representation is generated for each target field according to the field dictionary, the candidate value representation is extracted from the entity set and the table structure graph, the corresponding bilinear scoring is calculated to form a cost matrix, and the field value mapping and pairing confidence are obtained by optimal transport decoding under the mandatory constraint, uniqueness constraint and format template constraint. The entity relationship graph is constructed based on the entity set, the same paragraph adjacency and cross-block adjacency are set, the message passing and joint classification are performed to obtain the entity relationship triplets, the time standardization, unit conversion and document number structure analysis are performed, the cross-block deduplication and consistency check are performed according to the reading order and adjacent relationship, and the metadata and relationship data set are formed.

6. The archive information extraction and intelligent management system based on self-supervised learning according to claim 2, characterized in that, The standardized record library is taken as the base, the evidence link table and the strategy rule engine are taken as the basis, the permission control, the desensitization, the automatic archiving, the borrowing approval and the expiration reminder are implemented, the record correction is performed and the pseudo-label backflow is generated, and specifically, the method comprises the following steps: The standardized record library, the evidence link table, the original text mapping table and the index handle are read, the page coordinates, the block identifier and the character coordinates are obtained according to the record identifier; The permission control and the desensitization are performed by the strategy rule engine and the process processing, the visible field is cut according to the RBAC and the access control table, the desensitization is implemented according to the regular template, the sensitive word library and the coordinates, the automatic archiving and the borrowing approval are completed by using the directed state machine, the expiration reminder is generated according to the time template and the timer; The record identifier, the page coordinates, the block identifier and the character coordinates are returned according to the evidence link table, and the source field is outputted; The manual correction is recorded, the pseudo-label sample set is generated, the field values and the relationships are screened according to the consistency rules, the format templates and the threshold values, the evidence entry index is registered through the conflict filtering and the deduplication; The low-rank adaptation layer of the three-modal unified encoder model, the information extraction model and the structure-enhanced double-tower vector recall model is adjusted, the federated average aggregation and the differential privacy noise injection are adopted, the privacy budget and the model version are recorded, and the low-rank adaptation layer of the three-modal unified encoder model, the information extraction model and the structure-enhanced double-tower vector recall model is adjusted; The operation log and the audit track are written, and the new record, the pseudo-label sample, the model version number, the task result and the timestamp are associated to the original text mapping table.

Images