Semi-supervised learning method and system for correcting pseudo label bias based on document information

Abstract

The application discloses a pseudo-label bias correction semi-supervised learning method and system based on document information and belongs to the technical field of artificial intelligence. The method comprises the following steps: constructing a teacher model and a student model to share an encoder architecture; performing data transformation on unlabeled document samples respectively, inputting weak enhanced unlabeled samples into the teacher model, and outputting initial pseudo-labels; performing similarity matrix bias correction processing to generate guide pseudo-labels; constructing a double-branch adaptive alignment mechanism; constructing an overall joint loss function, and iteratively updating the student model weight based on the overall joint loss function through end-to-end back propagation; and loading the student model to perform forward reasoning on input documents to be extracted, output document entity information, complete document key information extraction, and realize optimal information extraction performance under the condition that the number of labeled samples is extremely small.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and more specifically, to a semi-supervised learning method and system for correcting pseudo-label bias based on document information. Background Technology

[0002] Document Key Information Extraction (DKIE) is a core task for automatically extracting key entity information from visual rich documents such as receipts, medical forms, and scientific papers, and it has significant commercial value in industries such as law, finance, and healthcare. Existing pre-training methods (LayoutLM, LayoutLMv2, BROS, etc.) perform well in fully supervised scenarios, but their core bottleneck is their dependence on a large amount of high-quality labeled data—in real-world business scenarios, collecting sufficient labeled samples for each type of entity is extremely costly. To alleviate the labeling burden, pseudo-labeled semi-supervised learning (PSSL) has become an important research direction: using the model's predictions of unlabeled data as "pseudo-labels" and incorporating them into the training process. Representative methods include: the fixed-threshold method FixMatch (using pseudo-labels only when the confidence exceeds a fixed threshold), the adaptive-threshold methods Dash and FlexMatch, and Simmatch and SoftMatch, which introduce consistency regularization.

[0003] Existing PSSL methods face two unique and serious technical obstacles when applied to DKIE tasks: 1. In DKIE, the same word may belong to drastically different entity categories in different document contexts—for example, "apple" refers to the brand (Apple Inc.) in a digital product receipt, but to food (apple) in a supermarket receipt, with completely opposite entity categories. Existing PSSL methods' thresholding mechanisms cannot detect this contextual dependency: fixed thresholds discard low-confidence but correct pseudo-labels; while adaptive thresholds can introduce more pseudo-labels, they inevitably introduce incorrect contextual ambiguity predictions.

[0004] 2. The DKIE benchmark dataset suffers from a double variance imbalance: on the one hand, entities of the same type exhibit vastly different formats across different documents (high intra-class variance), such as date entities which may display multiple formats like "YYYY-mm-DD" or "DD / mm / YYYY"; on the other hand, entities of different categories (such as invoice numbers, dates, and amounts) are highly similar visually and semantically (low inter-class variance). This is drastically different from the statistical characteristics of image classification datasets (CIFAR-10 / 100), leading to poor performance of existing general-purpose SSL methods on DKIE.

[0005] Therefore, there is an urgent need for a semi-supervised learning method and system based on document information to correct pseudo-label bias and solve the above problems. Summary of the Invention

[0006] The purpose of this invention is to provide a pseudo-label bias correction semi-supervised learning method and system based on document information, aiming to simultaneously solve the problems of low pseudo-label quality and intra-class / inter-class variance imbalance in DKIE semi-supervised learning, while achieving optimal information extraction performance with very few labeled samples.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a semi-supervised learning method for pseudo-label bias correction based on document information, comprising: S10. Construct a teacher-student dual-model shared encoder architecture, and use the shared encoder to extract document features from labeled document samples and unlabeled document samples respectively; the back end of the shared encoder is set with a classification head and an MLP projection head in parallel. S20. Perform weak and strong augmentation data transformations on the unlabeled document samples to obtain weakly augmented unlabeled samples and strongly augmented unlabeled samples; input the weakly augmented unlabeled samples into the teacher model and output the initial pseudo-labels. S30. Perform similarity matrix deviation correction processing to generate corrected guidance pseudo-labels; S40. Construct a two-branch adaptive alignment mechanism, calculate the intra-class alignment loss and inter-class alignment loss respectively, and obtain the total alignment loss: S50. Construct an overall joint loss function, which includes three parts: supervised classification loss, guided pseudo-label unsupervised loss, and weighted alignment loss; based on the overall joint loss, backpropagate the student model weights end-to-end in a round-robin fashion; in each iteration, use the exponential moving average (EMA) method to smoothly update the teacher model weights using the student model weights; S60. Load the student model to perform forward reasoning on the input document to be extracted, output the key entity information of the document, and complete the extraction of key information of the document.

[0008] Furthermore, S30 includes: S301. Using the predicted probability matrix output by the classification head of all labeled samples, construct the label pseudo-label similarity matrix and the label pseudo-label similarity matrix respectively. S302. The pseudo-label similarity matrix and the pseudo-label similarity matrix are weighted and fused using random mixing coefficients that follow a Gamma distribution to obtain the fused similarity matrix. The formula for calculating the fusion similarity matrix is: ; S303. Perform graph Laplacian symmetric normalization on the fused similarity matrix to obtain the normalized association matrix, where the fused similarity matrix is ​​obtained. The corresponding degree matrix The Laplace normalization formula for a symmetric graph is: ; S304. Apply the normalized correlation matrix to the initial pseudo-labels to generate guiding pseudo-labels that incorporate annotation context knowledge. The formula for generating guiding pseudo-labels is: ; in, To guide pseudo-tags.

[0009] Furthermore, step S301 includes: Let the predicted probability matrix of labeled samples be... The initial pseudo-label matrix of unlabeled samples is The probability matrix for strong enhancement prediction of unlabeled samples is: ; Formula for calculating pseudo-label similarity matrix: ; Formula for calculating pseudo-label similarity matrix: .

[0010] Furthermore, step S302 includes: Mixing coefficient Obeying Gamma Distribution, hyperparameters ; Formula for calculating the fusion similarity matrix: .

[0011] Furthermore, S40 includes: S401, Intra-class Alignment Branch: Maintain a fixed-length momentum queue grouped by entity category to store the historical predictions of labeled samples. Calculate the class-by-class similarity between the weakly enhanced and strongly enhanced document classification predictions and the corresponding class momentum queues. After sorting the similarity, apply an exponentially decaying weight. Give higher weights to difficult-to-correct samples with greater intra-class differences. Calculate the intra-class alignment loss using a weighted average. S402, Inter-class alignment branch: Map strong and weak enhancement document features to a non-linear representation space through the MLP projection head; calculate the positive similarity of strong and weak enhancement representations of the same sample, generate adaptive weights through exponential decay, and construct a weighted contrast loss as the inter-class alignment loss; S403. Add the intra-class alignment loss to the inter-class alignment loss to obtain the total loss of the bi-branch adaptive alignment.

[0012] Furthermore, S401 includes: Constructing a category momentum queue , For category Independent fixed-length storage queue, updated in batches using a first-in-first-out strategy; The formula for weakly enhanced prediction similarity is: ; The formula for strong enhancement prediction similarity is: ; The formula for calculating similarity is: ; The adaptive weight calculation formula is: ; ; in, For temperature hyperparameters, For temperature parameters, .

[0013] Furthermore, S401 also includes: The formula for intra-class alignment loss is: ; in, ; ; Only the strong enhancement branch participates in gradient backpropagation, while the weak enhancement branch is computed forward through the EMA teacher model.

[0014] Furthermore, S402 includes: Weakly enhanced nonlinear characterization is obtained through projection head mapping: ; ; The formula for calculating direct similarity is: ; The formula for calculating the weighted inter-class alignment loss is: .

[0015] S50 includes: The formula for calculating the overall joint loss is: ; in, This represents the total loss for bi-branch alignment. To balance the weighting coefficients, the optimal value is 0.2; The cross-entropy supervision loss for labeled samples, To enhance the distribution distance loss of prediction and guidance pseudo-labels.

[0016] This application also provides a pseudo-label bias correction semi-supervised learning system based on document information, which is used to perform a pseudo-label bias correction semi-supervised learning method for extracting key document information, including a feature extraction module, a teacher-student EMA training framework module, a similarity matrix bias correction module, an intra-class alignment module, an inter-class alignment module, and a joint optimization module. The feature extraction module has a built-in shared encoder, a classification head, and an MLP projection head. The shared encoder is used by both the teacher model and the student model to extract multimodal features from labeled and unlabeled documents. The classification head outputs the category probability distribution, and the MLP projection head outputs a nonlinear comparative representation. The teacher-student EMA training framework module is configured with a data augmentation unit, a pseudo-label generation unit, and a weight update unit. The data augmentation unit outputs weakly and strongly augmented text for unlabeled samples. The pseudo-label generation unit inputs the weakly augmented samples into the teacher model to obtain initial pseudo-labels. The weight update unit uses exponential moving average EMA to iteratively update the teacher model with the student model weights. The similarity matrix deviation correction module is connected to the teacher-student EMA training framework module and includes a dual matrix construction unit, a Gamma fusion unit, a graph Laplacian normalization unit, and a guided pseudo-label output unit. The dual matrix construction unit generates a labeled pseudo-label matrix and a labeled pseudo-label matrix. The Gamma fusion unit fuses the dual matrices based on Gamma distribution random coefficients. The graph Laplacian normalization unit normalizes the fused matrix. The guided pseudo-label output unit generates corrected guided pseudo-labels. The intra-class alignment module includes a built-in category momentum queue storage unit, a class-by-class similarity calculation unit, and an adaptive weighted loss unit. The momentum queue storage unit stores historical annotation predictions in groups according to entity categories. The similarity calculation unit calculates the similarity between the strong and weak enhancement predictions and historical samples of the same category. The adaptive weighted loss unit assigns high weights to intra-class hard-to-correct samples and outputs the intra-class alignment loss. The inter-class alignment module is connected to the MLP projection head of the feature extraction module and includes a nonlinear mapping unit and an adaptive weighted contrast loss unit. The nonlinear mapping unit transforms document features into an independent contrast representation space. The weighted contrast loss unit increases the representation distance between dissimilar samples, constrains the strength of the same sample to enhance representation consistency, and outputs the inter-class alignment loss. The joint optimization module receives supervised loss, unsupervised pseudo-label loss output by similarity matrix deviation correction, and intra-class / inter-class alignment loss, respectively, and weights them together to obtain the total loss, which is then used for backpropagation to update the student model parameters.

[0017] Compared with the prior art, the beneficial effects of the present invention are as follows: This application constructs a shared encoder architecture for teacher and student models, utilizing the shared encoder to extract features and ensuring consistency between the teacher and student models in the feature space. By setting a classification head and an MLP projection head in parallel at the back end of the shared encoder, the teacher and student models can simultaneously handle classification and representation learning tasks. Weak and strong augmentation transformations are performed on unlabeled document samples, and the teacher model generates initial pseudo-labels for the weakly augmented samples. Based on a consistency regularization strategy, the relatively stable prediction results of the teacher model can be used as the learning target of the student model, reducing noise from directly using the original data. Similarity matrix bias correction processing is performed, and the initial pseudo-labels are corrected by introducing contextual knowledge from labeled samples, solving the problem of pseudo-label errors caused by contextual ambiguity in the extraction of key document information, making the generated guiding pseudo-labels more reliable. By calculating intra-class alignment loss and inter-class alignment loss, constraints are imposed on the problems of large differences in entity format and high similarity between different categories of entities, respectively, enhancing the model's ability to discriminate entity features and effectively alleviating the double variance imbalance in the document dataset. A holistic joint loss function is constructed and combined with an EMA weight update strategy, which integrates supervised classification loss, guided pseudo-label unsupervised loss, and weighted alignment loss. The student model is iteratively updated through end-to-end backpropagation, and the teacher model is smoothly updated using EMA, ensuring the stability of the training process. This enables the model to learn efficiently from limited labeled data and a large amount of unlabeled data, and ultimately achieves accurate extraction of key document information through the student model. Attached Figure Description

[0018] The accompanying drawings, as part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application, but do not constitute an undue limitation of this application. Obviously, the drawings described below are merely some embodiments, and those skilled in the art can obtain other drawings based on these drawings without creative effort. In the drawings: Figure 1 A flowchart illustrating the implementation process of the pseudo-label bias correction semi-supervised learning method provided in this application; Figure 2 A flowchart illustrating the implementation of the similarity matrix deviation correction process provided in this application; Figure 3 A flowchart illustrating the implementation of the dual-branch adaptive alignment mechanism provided in this application; Figure 4 A schematic diagram illustrating the implementation process of the pseudo-label bias correction semi-supervised learning method provided in this application; Figure 5 The following diagram illustrates the intra-class and inter-class variance performance of the embodiments of this application in the FUNSD dataset scenario. Figure 6 A comparison chart of entity extraction visualization results on the CORD and FUNSD datasets for embodiments of this application provided in this application; Figure 7 The illustrations of SMBR and DBAA in the FUNSD 5% / 10% annotation scenario provided for the embodiments of this application; Figure 8 A schematic diagram of the confusion matrix on FUNSD and CORD for embodiments of this application provided for this application; Figure 9 Weighting factors provided for this application Impact curves on the performance of LayoutLM, LayoutLMv2, and BROS; Figure 10 The F1 convergence curves of this embodiment and other PSSL methods provided in this application in the 10% annotation scenario of FUNSD. Detailed Implementation

[0019] The technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0020] See Figures 1 to 10 As shown, a semi-supervised learning method for pseudo-label bias correction based on document information includes: S10. Construct a shared encoder architecture for the teacher model and student model, and use the shared encoder to extract document features from labeled document samples and unlabeled document samples respectively; the back end of the shared encoder is set with a classification head and an MLP projection head in parallel. S20. Perform weak enhancement and strong enhancement data transformations on the unlabeled document samples respectively to obtain weakly enhanced unlabeled samples and strongly enhanced unlabeled samples. Input the weakly enhanced unlabeled samples into the teacher model and output the initial pseudo-labels. S30. Perform similarity matrix deviation correction processing to generate corrected guidance pseudo-labels; S40. Construct a two-branch adaptive alignment mechanism, calculate the intra-class alignment loss and inter-class alignment loss respectively, and obtain the total alignment loss: S50. Construct an overall joint loss function, which includes supervised classification loss, guided pseudo-label unsupervised loss, and weighted alignment loss. Iteratively update the student model weights based on end-to-end backpropagation of the overall joint loss. In each iteration, the teacher model weights are smoothly updated using the student model weights through the exponential moving average (EMA) method. S60. Load the student model to perform forward reasoning on the input document to be extracted, output the document entity information, and complete the extraction of key information of the document.

[0021] In the above scheme, a shared encoder architecture is constructed for both the teacher and student models. This shared encoder extracts features, ensuring consistency between the teacher and student models in the feature space. By setting a classification head and an MLP projection head in parallel at the back end of the shared encoder, both models can simultaneously handle classification and representation learning tasks. Weak and strong augmentation transformations are performed on unlabeled document samples, and the teacher model generates initial pseudo-labels for the weakly augmented samples. Based on a consistency regularization strategy, the relatively stable prediction results of the teacher model can be used as the learning target for the student model, reducing noise from directly using the original data. Similarity matrix bias correction is performed, introducing contextual knowledge from labeled samples to correct the initial pseudo-labels, resolving the pseudo-label errors caused by contextual ambiguity in key document information extraction, making the generated guiding pseudo-labels more reliable. By calculating intra-class alignment loss and inter-class alignment loss, constraints are imposed to address the problems of large differences in entity format and high similarity between different categories of entities, respectively, enhancing the model's ability to discriminate entity features and effectively alleviating the double variance imbalance in the document dataset. A holistic joint loss function is constructed and combined with an EMA weight update strategy, which integrates supervised classification loss, guided pseudo-label unsupervised loss, and weighted alignment loss. The student model is iteratively updated through end-to-end backpropagation, and the teacher model is smoothly updated using EMA, ensuring the stability of the training process. This enables the model to learn efficiently from limited labeled data and a large amount of unlabeled data, and ultimately achieves accurate extraction of key document information through the student model.

[0022] Feasibility, setting A batch of labeled samples and their labels. This is a batch of unlabeled samples. and They share the same label space. The shared encoder is used by both the teacher and student models and extracts the following features: linear classification head. Output category probability MLP projector head Output nonlinear representation Weak enhancement was applied to the unlabeled samples. (Synonym replacement) and strong enhancement (Random insertion) yields the corresponding weakly or strongly enhanced representations. The teacher model generates initial pseudo-labels for the weakly enhanced unlabeled samples and obtains guiding pseudo-labels through similarity matrix bias correction (SMBR). The student model receives the strongly enhanced input and provides unsupervised supervision signals from the corrected guiding pseudo-labels. ), and the alignment loss is provided by the dual-branch adaptive alignment mechanism (DBAA). After each iteration, the teacher model weights are updated by the student model weights using an exponential moving average.

[0023] S30 includes: S301. Using the predicted probability matrix output by the classification head of all labeled samples, construct the label pseudo-label similarity matrix and the label pseudo-label similarity matrix respectively. S302, Using random mixing coefficients that follow a Gamma distribution. The similarity matrix between the labeled pseudo-labels and the labeled pseudo-labels is weighted and fused to obtain the fused similarity matrix. The formula for calculating the fusion similarity matrix is: ; S303. Perform graph Laplacian symmetric normalization on the fused similarity matrix to obtain the normalized association matrix, where the fused similarity matrix is ​​obtained. The corresponding degree matrix The Laplace normalization formula for a symmetric graph is: ; S304. Apply the normalized correlation matrix to the initial pseudo-labels to generate guiding pseudo-labels that incorporate annotation context knowledge. The formula for generating guiding pseudo-labels is: ; in, To guide pseudo-tags.

[0024] In the above scheme, a similarity matrix between labeled and pseudo-labels is constructed using all labeled samples. By explicitly calculating the similarity between samples, a reliable reference benchmark is provided for subsequent pseudo-label correction. The two matrices are weighted and fused using random mixing coefficients following a Gamma distribution. This introduces randomness and distribution characteristics, enabling the fusion process to dynamically balance the guiding role of labeled samples on pseudo-labels and enhance the model's adaptability to different sample distributions. Graph Laplacian symmetric normalization is performed on the fused similarity matrix, transforming the discrete similarity relationship into a smooth association matrix in a graph structure, giving the fused similarity matrix better numerical stability and propagation characteristics. The normalized association matrix is ​​applied to the initial pseudo-labels, and matrix multiplication explicitly injects the contextual knowledge of labeled samples into the pseudo-labels of unlabeled samples, realizing the correction process from initial prediction to guiding pseudo-labels. This correction process utilizes the connectivity of the graph structure, allowing pseudo-labels to adaptively adjust according to the distribution of labeled samples, thereby reducing pseudo-label errors caused by model prediction uncertainty and improving the quality and reliability of pseudo-labels in semi-supervised learning.

[0025] Step S301 includes: Let the predicted probability matrix of labeled samples be... The initial pseudo-label matrix of unlabeled samples is The probability matrix for strong enhancement prediction of unlabeled samples is: ; Formula for calculating pseudo-label similarity matrix: ; Formula for calculating pseudo-label similarity matrix: .

[0026] In the above scheme, a labeled pseudo-label similarity matrix is ​​constructed by multiplying the transpose of the labeled sample prediction probability matrix with the initial pseudo-label matrix of the unlabeled samples. This process aligns the known labeling information with the model's initial judgment of the unlabeled data, thereby capturing the semantic relationship between labeled samples and pseudo-labels. Simultaneously, a labeled pseudo-label similarity matrix is ​​constructed by multiplying the transpose of the labeled sample prediction probability matrix with the strongly enhanced prediction probability matrix of the unlabeled samples. By introducing strongly enhanced predictions, the model further strengthens its measurement of consistency between unlabeled samples and labeled samples under different enhancement perspectives.

[0027] Among them, for labeled samples Predictions obtained through a shared encoder For unlabeled samples Obtaining pseudo tags .

[0028] That is, the formula for the predicted probability matrix of labeled samples is: ; The formula for the initial pseudo-label matrix of unlabeled samples is: ; The formula for the strong enhancement prediction probability matrix of unlabeled samples is: .

[0029] This implementation method verifies the effect in a scenario based on the BROS model, and focuses on retaining the main effect table and the necessary implementation parameter table, as shown in Tables 1-2 below. Table 1. Main experimental results on the DKIE benchmark dataset based on BROS. Table 1 shows that the conclusions are consistent across the three basic models LayoutLM / LayoutLMv2 / BROS; they also achieve state-of-the-art performance on the XFUNSD multilingual dataset.

[0030] Table 2. Comparison of the performance of the SMBR module and the baseline in FUNSD and CORD (%) Table 2 shows that the similarity matrix bias correction module contributes 3.87% and 3.52% improvement in F1 scores on FUNSD and CORD, respectively, compared to the baseline. This not only verifies that the combination of the labeled pseudo-label similarity matrix and the labeled pseudo-label similarity matrix is ​​superior to a single matrix, but also demonstrates that using the predicted probability from the labeled data (soft label) as a correction is superior to using hard labels (GroundTruth), avoiding the situation where the predicted probability contains richer knowledge but also reduces the robustness to noise.

[0031] Step S302 includes: Mixing coefficient Obeying Gamma Distribution, hyperparameters ; Formula for calculating the fusion similarity matrix: .

[0032] In the above scheme, the semantic relevance between different unlabeled and labeled samples varies in the task of extracting key information from documents. By utilizing the randomness of the Gamma distribution, a suitable degree of uncertainty and diversity can be introduced into the fusion process, avoiding the accumulation of bias caused by a single fixed weight. Specifically, when the hyperparameter... When the Gamma distribution generates the mixing coefficient This allows the fusion process to flexibly balance the pseudo-label information of labeled samples with the original predicted pseudo-label information of unlabeled samples based on sample features. Furthermore, the weighted method based on the predicted probabilities of labeled data can more effectively utilize the contextual knowledge of labeled samples to calibrate the initial pseudo-labels of unlabeled samples. This dynamically optimizes the quality of pseudo-labels during training, enhances the model's adaptability to different document contexts, and effectively alleviates model training instability caused by pseudo-label bias.

[0033] Step S40 includes: S401, Intra-class Alignment Branch: Maintain a fixed-length momentum queue grouped by entity category to store the historical predictions of labeled samples. Calculate the class-by-class similarity between the classification predictions of weakly enhanced and strongly enhanced samples and the corresponding class momentum queues. After sorting the similarity, apply an exponentially decaying weight. Give higher weights to difficult-to-correct samples with greater intra-class differences. Calculate the intra-class alignment loss using a weighted average. S402, Inter-class alignment branch: Map strong and weak enhancement document features to a non-linear representation space through the MLP projection head; calculate the positive similarity of strong and weak enhancement representations of the same sample, generate adaptive weights through exponential decay, and construct a weighted contrast loss as the inter-class alignment loss; S403. Add the intra-class alignment loss to the inter-class alignment loss to obtain the total loss of the bi-branch adaptive alignment.

[0034] In the above scheme, the intra-class alignment branch maintains a fixed-length momentum queue grouped by entity category. It can use the prediction information of historical labeled samples as a reference benchmark, calculate the similarity between the current sample and historical samples of the same category, and apply exponential decay weights after similarity ranking. This enables automatic identification and weighting of difficult-to-correct samples, thereby effectively alleviating the intra-class variance problem caused by the diversity of document formats, and enabling the model to learn the core features of the same entity more robustly.

[0035] The inter-class alignment branch maps features to a non-linear representation space through an MLP projection head. It utilizes contrastive learning of strongly and weakly enhanced samples to calculate the head-to-head similarity of the same sample under different enhancements, and introduces adaptive weights to construct a contrastive loss. This effectively widens the distance between entities of different categories in the representation space while constraining the consistency of the same sample, thus solving the confusion problem caused by the high visual-semantic similarity between entities of different categories. By weightedly fusing the intra-class alignment loss and the inter-class alignment loss, the model enhances the structured representation of the feature space while ensuring the accuracy of entity classification.

[0036] Step S401 includes: Constructing a category momentum queue , For category Independent fixed-length storage queue, updated in batches using a first-in-first-out strategy; The formula for weakly enhanced prediction similarity is: ; The formula for strong enhancement prediction similarity is: ; The formula for calculating similarity is: ; The adaptive weight calculation formula is: ; ; in, For temperature hyperparameters, For temperature parameters, .

[0037] In the above scheme, by introducing a category momentum queue mechanism, an independent fixed-length storage space is maintained for each type of entity, and dynamically updated using a first-in-first-out (FIFO) strategy. This ensures that the model can continuously acquire and utilize the prediction information of historical labeled samples, thus providing a stable reference benchmark for the intra-class alignment branch of the current sample. By calculating the similarity between the weak and strong enhancement prediction results and the category momentum queue, the model can quantify the degree of consistency between the current prediction and historical samples of the same class. In addition, introducing a temperature hyperparameter T to scale the similarity can effectively smooth the probability distribution and enhance the model's sensitivity to similarity differences. By ranking the similarity and applying exponentially decaying adaptive weights, the model can automatically identify and focus on those difficult-to-correct samples with low similarity.

[0038] S401 also includes: The formula for intra-class alignment loss is: ; in, ; ; Only strong enhancement branches participate in gradient backpropagation, while weak enhancement branches are computed forward through the EMA teacher model.

[0039] It should be noted that when it is necessary to conserve GPU memory, gradients are only output through strong enhancement. Backpropagation occurs when the weakly enhanced branch propagates forward through the EMA teacher model.

[0040] In the above scheme, the intra-class alignment loss is decomposed into the sum of the weakly enhanced sample loss and the strongly enhanced sample loss, which respectively measure the matching degree between the sample and the class momentum queue under different enhancement intensities. The model can force strongly enhanced samples to converge towards the stable pseudo-label distribution provided by the teacher model, while avoiding excessive noise interference introduced by the weakly enhanced branch in backpropagation. Thus, it ensures the robustness of the model to the learning of entity class features, while effectively improving the convergence efficiency and prediction accuracy of semi-supervised learning in the task of extracting key information from documents.

[0041] S402 includes: Weakly enhanced nonlinear characterization is obtained through projection head mapping: ; ; The formula for calculating direct similarity is: ; The formula for calculating the weighted inter-class alignment loss is: .

[0042] In the above scheme, firstly, the original document features extracted by the shared encoder are mapped to a nonlinear representation space using an MLP projection head. This breaks the linear constraints of the original feature space, enabling the model to capture the semantic features of document entities from a more abstract dimension, thus providing a more discriminative representation foundation for subsequent inter-class differentiation. Furthermore, by calculating the head-to-head similarity between weakly and strongly enhanced samples in the nonlinear space, the consistency of the same sample under different enhancement transformations can be effectively constrained, ensuring the model's robustness in extracting information about the same entity. A weighted inter-class alignment loss function is introduced. By applying adaptive weights to different sample pairs, the focus of contrastive learning can be dynamically adjusted. While widening the representation distance between dissimilar samples, the constraint on key sample pairs is strengthened, thereby solving the confusion problem caused by small inter-class variance in the extraction of key document information.

[0043] S50 includes: The formula for calculating the overall joint loss is: ; in, This represents the total loss for bi-branch alignment. To balance the weighting coefficients, the optimal value is 0.2; The cross-entropy supervision loss for labeled samples, For cross-entropy loss, The boot pseudo-tag generated for SMBR To enhance forecasting; ; To enhance the distribution distance loss of prediction and guidance of false labels; .

[0044] In the above scheme, a balancing weight coefficient is introduced. The model dynamically adjusts the proportion of the bi-branch alignment loss in the overall loss, thus establishing an effective balance between supervised and unsupervised learning. Supervised classification loss serves as the foundation, ensuring the model's ability to identify key entity information; unsupervised loss strengthens the model's efficiency in utilizing unlabeled data by enhancing the distribution distance constraint between the predicted and guided pseudo-labels; and the bi-branch alignment loss further improves the model's feature discrimination in complex document environments through intra-class and inter-class alignment constraints. This joint loss function construction method allows the model to fine-tune its parameters under a unified optimization objective through end-to-end backpropagation, effectively resolving training conflicts in multi-objective tasks and improving the overall performance of key document information extraction.

[0045] Table 2 shows that the overall joint loss contributes 9.29% and 7.28% to the F1 score improvement in FUNSD and CORD, respectively. t-SNE visualization shows that the introduction of the overall joint loss (DBAA) results in more compact cluster structures and clearer inter-cluster boundaries. Furthermore, Table 2 indicates that the contribution of the intra-cluster alignment branch is greater than that of the inter-cluster alignment branch.

[0046] The shared encoder is any one of LayoutLM, LayoutLMv2, or BROS visual document pre-trained encoder.

[0047] In the above scheme, by clearly defining the specific selection of the shared encoder, the foundation for feature extraction of the entire semi-supervised learning framework is laid. LayoutLM, LayoutLMv2, and BROS are all pre-trained models specifically designed for visual rich documents, capable of simultaneously processing the textual semantic information and spatial layout information of documents. By using LayoutLM, LayoutLMv2, or BROS as the shared encoder, it can be ensured that both the teacher model and the student model can extract document features containing rich contextual relationships when processing labeled and unlabeled document samples. This selection not only ensures the professionalism of feature extraction but also provides a highly discriminative representation space for subsequent steps based on similarity matrix-based bias correction and intra-class and inter-class alignment mechanisms through multimodal fusion capabilities, thereby effectively supporting the model's ability to extract key information in complex document scenarios.

[0048] Example 1: Understanding FUNSD Forms (5% annotation) Input labeled and unlabeled forms, extract text + layout fusion features through BROS shared encoder, and output two branches in parallel on the backend: classification head: outputs the probability distribution of 4 types of entities; MLP projection head: outputs nonlinear representations for contrastive learning.

[0049] Dual-intensity data augmentation for unlabeled samples Weak enhancement: synonym replacement (low perturbation, fed into the teacher model to generate original pseudo-labels); Strong enhancement: Random text insertion (high perturbation, fed into the student model for training).

[0050] The teacher model infers weakly augmented unlabeled samples, resulting in initial pseudo-labels with contextual ambiguity and noise. Similarity Matrix Bias Correction (SMBR) dual-matrix construction; ;Predicted probability of labeled sample × unlabeled original pseudo-label; ;Predicted probability of labeled samples × Strongly enhanced prediction of unlabeled samples; Mixing coefficient Gamma (0.6, 0.6) is the dynamically weighted fusion similarity matrix bias correction (SMBR) matrix. Graph Laplace symmetric normalization eliminates noise from neighboring nodes, yielding the symmetric graph Laplace normalization formula. ; Generate guide pseudo-tags By injecting contextual distribution knowledge from a small number of labeled samples, the ambiguity of the same type of "apple" can be resolved.

[0051] Maintain four independent momentum queues for entities and store seven historical predictions of labeled samples; calculate the cosine similarity between strong and weak enhancement predictions and the corresponding category queues; apply exponential decay weights to the similarity in ascending order, and give higher loss weights to difficult samples with large format differences (such as different dates and different answer fields) to compress the representation distance of similar entities. The projection head generates nonlinear representations and constructs an adaptive weighted contrast loss to widen the boundaries of four types of visually and semantically similar entities (question, header, answer, and other), thus solving the problem of low variance confusion between DKIE classes. The total alignment loss obtained by merging is: .

[0052] Overall joint loss: ; in, Cross-entropy supervision loss based on only 7 labeled samples; Student-enhanced prediction and SMBR-corrected distribution loss of pseudo-labels; Balanced alignment loss weights; In each iteration, the teacher model is smoothly updated using the student model weights EMA to stabilize the quality of pseudo-label generation.

[0053] After training, the teacher model is discarded, and only the student model performs entity extraction on 50 forms in the test set, outputs entity category predictions, and calculates the F1 score.

[0054] In the above embodiments, the performance of the F1 index at the suboptimal baseline and the complete PLBR (SMBR+DBAA) in this application is compared as follows: Baseline FreeMatch (same as BROS base, 5% marking): F1=62.13% Complete PLBR (SMBR+DBAA): F1=64.24%; As can be seen, the implementation method of this application improves the contribution of FUNSD by +2.11% compared with the suboptimal baseline.

[0055] The following is a comparison of the F1 score performance in pure baseline (without SMBR, without DBAA) and dual-branch SMBR + full DBAA: Pure baseline: 62.18%; SMBR only: 66.05%; Complete DBAA with two branches: 71.47%; As can be seen, the implementation method of this application improves the contribution of FUNSD by +9.29% compared with the baseline.

[0056] Example 2: CORD Receipt Entity Extraction (10% Labeling) Input: 10% of the 800 training receipts are labeled, and the remaining 720 are unlabeled; 100 test sets; 4 categories / 30 fields.

[0057] Results: F1 score reached 83.70% (BROS base model), an improvement of 2.59% over the suboptimal FreeMatch method; the early PLBR guidance pseudo-label quality advantage made the convergence speed significantly faster than all baselines (about 30 iterations).

[0058] Example 3: XFUNSD Multilingual Relation Extraction (10% Annotation) Input: 1393 forms in 7 languages ​​(Chinese / Japanese / Spanish / French / Italian / German / Portuguese), 10% of which are labeled; the tasks are semantic entity recognition (SER) and relation extraction (RE).

[0059] Results: SER reached 66.56% and RE reached 42.77%, both of which are better than all state-of-the-art semi-supervised baselines (second best FreeMatch: SER 64.78%, RE 41.09%).

[0060] This application also provides a document-information-based pseudo-label bias correction semi-supervised learning system for executing a document-information-based pseudo-label bias correction semi-supervised learning method, including a feature extraction module, a teacher-student EMA training framework module, a similarity matrix bias correction module, an intra-class alignment module, an inter-class alignment module, and a joint optimization module; The feature extraction module has a built-in shared encoder, a classification head, and an MLP projection head. The shared encoder is used by both the teacher model and the student model to extract multimodal features from labeled and unlabeled documents. The classification head outputs the category probability distribution, and the MLP projection head outputs a nonlinear comparative representation. The teacher-student EMA training framework module is configured with a data augmentation unit, a pseudo-label generation unit, and a weight update unit. The data augmentation unit outputs weakly and strongly augmented text for unlabeled samples. The pseudo-label generation unit inputs the weakly augmented samples into the teacher model to obtain initial pseudo-labels. The weight update unit uses exponential moving average EMA to iteratively update the teacher model with the student model weights. The similarity matrix deviation correction module is connected to the teacher-student EMA training framework module and includes a dual matrix construction unit, a Gamma fusion unit, a graph Laplacian normalization unit, and a guided pseudo-label output unit. The dual matrix construction unit generates a labeled pseudo-label matrix and a labeled pseudo-label matrix. The Gamma fusion unit fuses the dual matrices based on Gamma distribution random coefficients. The graph Laplacian normalization unit normalizes the fused matrix. The guided pseudo-label output unit generates corrected guided pseudo-labels. The intra-class alignment module includes a built-in category momentum queue storage unit, a class-by-class similarity calculation unit, and an adaptive weighted loss unit. The momentum queue storage unit stores historical annotation predictions in groups according to entity categories. The similarity calculation unit calculates the similarity between the strong and weak enhancement predictions and historical samples of the same category. The adaptive weighted loss unit assigns high weights to intra-class hard-to-correct samples and outputs the intra-class alignment loss. The inter-class alignment module is connected to the MLP projection head of the feature extraction module and includes a nonlinear mapping unit and an adaptive weighted contrast loss unit. The nonlinear mapping unit transforms document features into an independent contrast representation space. The weighted contrast loss unit increases the representation distance between dissimilar samples, constrains the strength of the same sample to enhance representation consistency, and outputs the inter-class alignment loss. The joint optimization module receives supervised loss, unsupervised pseudo-label loss output by similarity matrix deviation correction, and intra-class / inter-class alignment loss, respectively, and weights them together to obtain the total loss, which is then used for backpropagation to update the student model parameters.

[0061] In the above scheme, the feature extraction module ensures that the teacher and student models interact in a unified feature space through a shared encoder. The parallel setting of the classification head and the MLP projection head provides multi-dimensional feature support for subsequent classification and contrastive learning tasks.

[0062] The teacher-student EMA training framework module utilizes the exponential moving average mechanism to achieve smooth updates of teacher model weights, providing a foundation for generating stable initial pseudo-labels. The difference between strong and weak augmentation introduced by the data augmentation unit provides necessary perturbation samples for subsequent bias correction and alignment.

[0063] The similarity matrix bias correction module constructs a similarity matrix between labeled samples, pseudo-labels, and unlabeled samples, and fuses these matrices using Gamma-distributed random coefficients, effectively capturing the contextual relationships between documents. Through graph Laplacian normalization, prior knowledge from the labeled samples is incorporated into the pseudo-labels, thereby correcting the contextual ambiguity in the initial pseudo-labels and generating more guiding pseudo-labels.

[0064] The intra-class alignment module stores historical predictions through a category momentum queue, enabling it to dynamically perceive the distribution characteristics of entity categories. By assigning higher weights to intra-class hard-to-correct samples, it forces the model to focus on entities that exhibit high variance across different documents, thereby improving the model's robustness to intra-class discrepancies.

[0065] The inter-class alignment module uses an MLP projection head to map features to a non-linear space. By contrastive learning, it widens the distance between dissimilar samples and constrains the consistency of representation of the same sample under different enhancements. This effectively alleviates the confusion caused by the high visual and semantic similarity of different types of entities.

[0066] The joint optimization module weighted and fused the losses generated by the above modules. Through end-to-end backpropagation, it achieved coordinated optimization of classification accuracy, pseudo-label reliability and feature space alignment. This ensured that even in the absence of a large amount of labeled data, the model could still efficiently extract key information from documents through in-depth mining and bias correction of unlabeled data.

[0067] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A semi-supervised learning method based on document information for pseudo-label bias correction, characterized in that, include: S10. Construct a shared encoder architecture for teacher and student models. Use the shared encoder to extract document features from labeled and unlabeled document samples respectively. The back end of the shared encoder is set with a classification head and an MLP projection head in parallel. S20. Perform weak enhancement and strong enhancement data transformations on the unlabeled document samples respectively to obtain weakly enhanced unlabeled samples and strongly enhanced unlabeled samples. Input the weakly enhanced unlabeled samples into the teacher model and output the initial pseudo-labels. S30. Perform similarity matrix deviation correction processing to generate corrected guidance pseudo-labels; S40. Construct a dual-branch adaptive alignment mechanism, calculate the intra-class alignment loss and inter-class alignment loss respectively, and obtain the total alignment loss; S50. Construct an overall joint loss function, which includes supervised classification loss, guided pseudo-label unsupervised loss, and weighted alignment loss. Iteratively update the student model weights based on end-to-end backpropagation of the overall joint loss. In each iteration, the teacher model weights are smoothly updated using the student model weights through the exponential moving average (EMA) method. S60. Load the student model to perform forward reasoning on the input document to be extracted, output the document entity information, and complete the extraction of key information of the document.

2. The pseudo-label bias correction semi-supervised learning method for extracting key document information according to claim 1, characterized in that, Step S30 includes: Step S301: Using the predicted probability matrix output by the classification head of all labeled samples, construct the label pseudo-label similarity matrix and the label pseudo-label similarity matrix respectively; Step S302: The pseudo-label similarity matrix and the pseudo-label similarity matrix are weighted and fused using random mixing coefficients that follow a Gamma distribution to obtain a fused similarity matrix. The formula for calculating the fusion similarity matrix is: ； Step S303: Perform graph Laplacian symmetric normalization on the fused similarity matrix to obtain the normalized association matrix, wherein the fused similarity matrix is ​​obtained. The corresponding degree matrix The Laplace normalization formula for a symmetric graph is: ； Step S304: Apply the normalized association matrix to the initial pseudo-labels to generate guiding pseudo-labels that incorporate annotation context knowledge. The formula for generating guiding pseudo-labels is: ； in, To guide pseudo-tags.

3. The pseudo-label bias correction semi-supervised learning method for extracting key document information according to claim 2, characterized in that, Step S301 includes: Let the predicted probability matrix of labeled samples be... The initial pseudo-label matrix of unlabeled samples is The probability matrix for strong enhancement prediction of unlabeled samples is: ; Formula for calculating pseudo-label similarity matrix: ; Formula for calculating pseudo-label similarity matrix: 。 4. The pseudo-label bias correction semi-supervised learning method for extracting key document information according to claim 2, characterized in that, Step S302 includes: Mixing coefficient Obeying Gamma Distribution, hyperparameters ; Formula for calculating the fusion similarity matrix: 。 5. The pseudo-label bias correction semi-supervised learning method for extracting key document information according to claim 1, characterized in that, Step S40 includes: S401, Intra-class Alignment Branch: Maintain a fixed-length momentum queue grouped by entity category to store the historical predictions of labeled samples. Calculate the class-by-class similarity between the classification predictions of weakly enhanced and strongly enhanced samples and the corresponding class momentum queues. After sorting the similarity, apply an exponentially decaying weight. Give higher weights to difficult-to-correct samples with greater intra-class differences. Calculate the intra-class alignment loss using a weighted average. S402, Inter-class alignment branch: Map strong and weak enhancement document features to a non-linear representation space through the MLP projection head; calculate the positive similarity of strong and weak enhancement representations of the same sample, generate adaptive weights through exponential decay, and construct a weighted contrast loss as the inter-class alignment loss; S403. Add the intra-class alignment loss to the inter-class alignment loss to obtain the total loss of the bi-branch adaptive alignment.

6. The pseudo-label bias correction semi-supervised learning method for extracting key document information according to claim 5, characterized in that, Step S401 includes: Constructing a category momentum queue , For category Independent fixed-length storage queue, updated in batches using a first-in-first-out strategy; The formula for weakly enhanced prediction similarity is: ； The formula for strong enhancement prediction similarity is: ； The formula for calculating similarity is: ； The adaptive weight calculation formula is: ； ； in, For temperature hyperparameters, For temperature parameters, .

7. The pseudo-label bias correction semi-supervised learning method for extracting key document information according to claim 5, characterized in that, Step 401 further includes: The formula for intra-class alignment loss is: ； in, ; ； Only the strong enhancement branch participates in gradient backpropagation, while the weak enhancement branch is computed forward through the EMA teacher model.

8. The pseudo-label bias correction semi-supervised learning method for extracting key document information according to claim 5, characterized in that, Step S402 includes: Weakly enhanced nonlinear characterization is obtained through projection head mapping: ； ； The formula for calculating direct similarity is: ； The formula for calculating the weighted inter-class alignment loss is: 。 9. The pseudo-label bias correction semi-supervised learning method for extracting key document information according to claim 1, characterized in that, Step S50 includes: The formula for calculating the overall joint loss is: ； in, This represents the total loss for bi-branch alignment. To balance the weighting coefficients, the optimal value is 0.2; The cross-entropy supervision loss for labeled samples, To enhance the distribution distance loss of prediction and guidance pseudo-labels.

10. A semi-supervised learning system for correcting pseudo-label bias based on document information, used to execute the semi-supervised learning method according to any one of claims 1-9, characterized in that, It includes a feature extraction module, a teacher-student EMA training framework module, a similarity matrix deviation correction module, an intra-class alignment module, an inter-class alignment module, and a joint optimization module; The feature extraction module has a built-in shared encoder, classification head, and MLP projection head; the shared encoder is used by both the teacher model and the student model to extract multimodal features from labeled and unlabeled documents. The classification head outputs the category probability distribution, and the MLP projection head outputs a nonlinear comparative representation. The teacher-student EMA training framework module is configured with a data augmentation unit, a pseudo-label generation unit, and a weight update unit. The data augmentation unit outputs weakly and strongly augmented text for unlabeled samples. The pseudo-label generation unit inputs the weakly augmented samples into the teacher model to obtain initial pseudo-labels. The weight update unit uses exponential moving average EMA to iteratively update the teacher model with the student model weights. The similarity matrix deviation correction module is connected to the teacher-student EMA training framework module and includes a dual matrix construction unit, a Gamma fusion unit, a graph Laplacian normalization unit, and a guided pseudo-label output unit. The dual matrix construction unit generates a labeled pseudo-label matrix and a labeled pseudo-label matrix. The Gamma fusion unit fuses the dual matrices based on Gamma distribution random coefficients. The graph Laplacian normalization unit normalizes the fused matrix. The guide pseudo-tag output unit generates the corrected guide pseudo-tag; The intra-class alignment module includes a built-in category momentum queue storage unit, a class-by-class similarity calculation unit, and an adaptive weighted loss unit. The momentum queue storage unit stores historical annotation predictions in groups according to entity categories. The similarity calculation unit calculates the similarity between the strong and weak enhancement predictions and historical samples of the same category. The adaptive weighted loss unit assigns high weights to intra-class hard-to-correct samples and outputs the intra-class alignment loss. The inter-class alignment module is connected to the MLP projection head of the feature extraction module and includes a nonlinear mapping unit and an adaptive weighted contrast loss unit. The nonlinear mapping unit transforms document features into an independent contrast representation space. The weighted contrast loss unit increases the representation distance between dissimilar samples, constrains the strength of the same sample to enhance representation consistency, and outputs the inter-class alignment loss. The joint optimization module receives supervised loss, unsupervised pseudo-label loss output by similarity matrix deviation correction, intra-class alignment loss, or inter-class alignment loss, respectively, and weights them to obtain the total loss, which is then used for backpropagation to update the student model parameters.

Images