A label noise detection method based on multi-time step loss sequence

Abstract

The application discloses a label noise detection method based on a multi-time step loss sequence, utilizes a character recognition module to recognize characters from an input image; utilizes an MSL-MentorNet label noise detection module to analyze a loss sequence of the input image in a continuous training period according to a character recognition result, dynamically allocates sample weights to detect and filter noise labels, and feeds back screened data to a character recognition module to optimize model training. The application adopts an MSL-MentorNet label noise detection and noise filtering mechanism based on dynamic curriculum learning, analyzes a loss sequence of a sample in a continuous training period, utilizes a BiLSTM network to distinguish noise labels from clean difficult samples, and combines a linear scheduling strategy to gradually increase a training data amount. Based on the method of the application, the complex text image recognition accuracy can be improved, and the method is especially suitable for robust recognition and noise filtering of complex text images in scenarios such as ancient book digitization, historical document restoration, industrial OCR quality inspection and the like.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of computer vision and natural language processing, and specifically relates to a label noise detection method based on multi-time step loss sequences. Background Technology

[0002] In the intersection of computer vision and natural language processing, text recognition technology is widely used. However, labeled data often contains noise. Traditional text recognition models rely on large amounts of labeled data for training, and the manual labeling process is prone to introducing noisy labels due to subjective errors and inconsistent standards. These noisy labels interfere with model training, causing the model to learn incorrect patterns and reducing recognition accuracy and generalization ability.

[0003] Existing methods for addressing label noise have several limitations. While manual relabeling can remove noise, it is time-consuming, labor-intensive, and costly when dealing with large-scale datasets. Statistical analysis-based methods, such as filtering noisy labels by calculating sample loss statistics, struggle to handle complex noise patterns and suffer from poor accuracy. Deep learning-based methods, although attempting to design specific network structures to learn noise features, often rely heavily on prior knowledge, resulting in complex network designs and insufficient generalization across different datasets and noise types.

[0004] Especially for Mongolian, as a phonetic script, the different positions of letters in a word result in different spellings, and there are no obvious intervals between words. This increases the difficulty of character segmentation and feature extraction for Mongolian text images, and existing label noise detection methods are difficult to adapt to its characteristics. Summary of the Invention

[0005] In order to overcome the shortcomings of the prior art, the present invention aims to provide a label noise detection method based on multi-time step loss sequence to improve the accuracy and reliability of Mongolian character recognition.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] A label noise detection method based on multi-timestep loss sequences includes the following steps:

[0008] Step 1: Use the text recognition module to recognize text from the input image;

[0009] Step 2: Using the MSL-MentorNet label noise detection module, based on the text recognition results, analyze the loss sequence of the input image in continuous training cycles, dynamically allocate sample weights to detect and filter noise labels, and feed the filtered data back to the text recognition module to optimize model training.

[0010] In one embodiment, the text recognition module performs geometric correction, visual feature extraction, and sequence decoding on the input image to obtain the text recognition result; the geometric correction is achieved by correcting curved and tilted text through a thin plate spline transformation layer, which utilizes a differentiable mesh to achieve precise image transformation.

[0011] In one embodiment, the text recognition module includes:

[0012] The thin-plate spline transformation layer predicts the reference points of the input image through differentiable matrix operations, calculates the TPS transformation parameters based on the reference points and the fixed base-reference points, generates a differentiable grid, and generates a corrected regular text image through interpolation. The fixed base-reference points are a set of fixed coordinate points pre-set during the image correction process, which are used to provide a standardized reference frame for curved or tilted text images so that the text regions in the original image can be corrected into regular rectangular shapes through thin-plate spline (TPS) transformation.

[0013] The ResNet feature extraction layer extracts multi-scale visual features from the corrected regular text image;

[0014] A bidirectional LSTM layer is used to perform contextual modeling on the feature sequence;

[0015] The attention decoding layer, combined with the implicit language model, decodes the feature sequence into a character sequence, which is the text recognition result.

[0016] In one embodiment, the implementation of the thin-plate spline conversion layer includes:

[0017] Step 1: Predict K reference points of the input image using a localization network;

[0018] Step 2: Using a mesh generator, based on the reference point and the fixed base-reference point, calculate the TPS transformation parameters through differentiable matrix operations to generate a differentiable mesh;

[0019] Step 3: Based on the differentiable grid, the sampler is used to perform pixel mapping and weighted average calculation through bilinear interpolation to generate the corrected regular text image. The bilinear interpolation performs gradient calculation during backpropagation to optimize the thin plate spline transformation layer in the neural network training.

[0020] In one embodiment, the attention decoding layer employs a beam search strategy to generate character sequences through the following steps:

[0021] Step 1: Calculate the weighted sum of the semantic vectors as the encoder's hidden states;

[0022] Step 2: Combine the predicted character embedding from the previous time step and update the decoding state using LSTM;

[0023] Step 3: Output the character probability distribution at the current time step.

[0024] In one embodiment, the MSL-MentorNet tag noise detection module includes:

[0025] The loss sequence generation unit records the loss value of each input image in consecutive training cycles, constructs a loss sequence, and analyzes the fluctuation characteristics and outliers of the loss sequence to initially detect samples that may have noisy labels, providing a basis for subsequent processing.

[0026] The BiLSTM discriminative network takes the loss sequence, label type, and training progress information as input and outputs sample weights. The weights reflect the probability that a sample is a noise sample, thus enabling the detection and differentiation of noise-labeled samples.

[0027] The linear scheduling unit dynamically adjusts the proportion of input images participating in training based on the preset noise rate and the sample weights output by the BiLSTM discriminant network, thereby optimizing the training data.

[0028] In one embodiment, the loss sequence generation unit constructs the loss sequence as follows: Let the time step in the training process be t, t = 1, 2, ..., T, where T is the total number of time steps, and the number of consecutive training cycles be n, n = 1, 2, ..., N, where N is the total number of training cycles. For the i-th input image I... i :

[0029] At each time step within each training cycle, each image is input into the text recognition module and trained against the MSL-MentorNet label noise detection module. During each training iteration, the loss value is calculated based on the model's prediction results and the true labels, typically using common loss calculation methods such as the cross-entropy loss function. Where C is the number of categories, y i,j Let j be the value of the j-th class in the real label. Predict the probability of class j for the model;

[0030] During consecutive training cycles, the loss value is recorded sequentially at time steps to construct the loss sequence S. i ={L i,1,1 ,L i,1,2 ,...,L i,1,T ,L i,2,1 ,L i,2,2 ,...,L i,2,T ,...,L i,N,1 ,L i,N,2 ,...,L i,N,T This is to illustrate the loss change for each image under multiple time steps and continuous training cycles, where i = 1, 2, ..., M, and M is the total number of input images.

[0031] In one embodiment, the BiLSTM discrimination network is trained through the following process:

[0032] Step 1, synthesize IDN and RCN noise data on the clean subset; the clean subset refers to a part of the data selected from the original dataset, with accurate annotations and no noise labels;

[0033] Step 2, taking the loss sequence, label type, and training cycle percentage as inputs and the binary weights as outputs, minimize the mean square error;

[0034] Step 3, update the network parameters through the Adam optimizer.

[0035] In one embodiment, the method for synthesizing IDN noise data on the clean subset is as follows:

[0036] 1), Determine the noise ratio: Set the IDN noise rate ρIDN, which represents the proportion of samples to add feature-related noise in the clean subset;

[0037] 2), Select samples: Randomly select nIDN = ρIDN × P samples from the clean subset, where P is the total number of samples in the clean subset;

[0038] 3), Analyze the sample features: For each selected sample, analyze its features. Taking the text recognition image sample as an example, features such as the stroke structure and grayscale distribution of the image can be analyzed;

[0039] 4), Modify the label according to the feature association: Modify the label according to the association relationship between the sample features and the features of other categories. For example, for some text images with similar stroke structures, if a sample is "己", according to the similarity of its features to "已", modify its label to "已", so as to generate noise labels related to the sample features and complete the synthesis of IDN noise data;

[0040] The method for synthesizing RCN noise data on the clean subset is as follows:

[0041] 1), Determine the noise ratio: Set the RCN noise rate ρRCN, which represents the proportion of samples to add random category noise in the clean subset; for example, if ρRCN = 0.2, it means that 20% of the samples in the clean subset will be added with RCN noise.

[0042] 2), Select samples: Randomly select nRCN = ρRCN × P samples from the clean subset.

[0043] 3), Modify the labels: For each selected sample, randomly replace its true label with the label of other classes in the dataset; for example, in an optical character recognition task, if the true label of a sample is "Hello", randomly replace it with other text category labels such as "Goodbye". Complete the synthesis of RCN noise data.

[0044] In one embodiment, the linear scheduling unit adjusts the amount of data participating in training according to the following rules:

[0045] Step 1, only select high-weight samples to participate in training in the initial stage; the high-weight samples refer to the samples that are determined to be more reliable and more likely to be correctly labeled after being processed by the BiLSTM discriminant network in the MSL-MentorNet label noise detection module;

[0046] Step 2, linearly increase the sample retention rate with the training cycle until it reaches the preset 1 - ε, where ε is the noise rate.

[0047] Compared with the prior art, the present invention addresses two major challenges in complex text image recognition - geometric deformation correction and label noise interference. It adopts the MSL-MentorNet label noise detection and the noise filtering mechanism based on dynamic curriculum learning. By analyzing the loss sequence of samples in consecutive training cycles, it uses the BiLSTM network to distinguish noise labels from clean difficult samples, and combines the linear scheduling strategy to gradually increase the amount of training data. Based on the method of the present invention, the accuracy of complex text image recognition can be improved, especially suitable for the robust recognition and noise filtering of complex text images in scenarios such as ancient book digitization, historical document restoration, and industrial OCR quality inspection. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is a schematic diagram of the line-level optical character recognition framework.

[0049] Figure 2 It is a schematic diagram of the reference points and TPS transformation in the text line picture, where the left figure is the input image I and the right figure is the input image I ′ .

[0050] Figure 3 It is a schematic diagram of the BiLSTM network structure and training process of MSL-MentorNet.

[0051] Figure 4 It is a schematic diagram of the test accuracy curve of label noise detection.

[0052] Figure 5 It is a test curve graph of Test Accuracy - epoch on the augmented dataset. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0053] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings and examples.

[0054] Mongolian script, as a phonetic script, has unique structural characteristics. The letters in a word have different spellings depending on their position in the word, and there are no obvious intervals between words. This not only makes character segmentation difficult, but also makes feature extraction more complex, thereby increasing the probability of generating noisy labels.

[0055] Therefore, this invention provides a label noise detection method based on a multi-time-step loss sequence to support complex text image recognition, such as... Figure 1 As shown, it mainly includes the following steps:

[0056] Step 1: Using the text recognition module, geometric correction, visual feature extraction, and sequence decoding are performed on the input image to recognize text from the input image.

[0057] This step integrates multiple technologies to improve recognition performance. Specifically, a Thin Plate Spline Transformation (TPS) layer is used to predict the reference points of the input image and adaptively correct the geometric deformation of curved and tilted text, generating a corrected regular text image through interpolation. A ResNet deep convolutional network is used as the feature extraction layer to extract multi-scale visual features (deep features), including local and global semantic features, from the regular text image, enhancing the perception of blurred and bleed characters. A bidirectional LSTM layer is used to model the contextual dependencies of long text lines, capturing long-distance context and solving the problems of broken characters and ink spread in early lead-type newspapers. An attention decoding layer is introduced, which integrates an implicit language model to decode the feature sequence into a character sequence, eliminating semantic ambiguity and improving the recognition accuracy of easily confused characters such as blurred and bleed characters (e.g., Mongolian word root variants). These technologies work synergistically, from geometric correction to feature extraction and semantic processing, comprehensively contributing to the accurate recognition of complex text images.

[0058] The text recognition process of this invention can be specifically described as follows:

[0059] Step 11, Text line segmentation: Locate text lines based on reprojection contour analysis and Gaussian filtering.

[0060] In one text recognition embodiment, this step specifically includes:

[0061] (1) Input the original image (size: 3000×4000 pixels, resolution 300dpi).

[0062] (2) Image preprocessing

[0063] Binarization: The image is converted into a black-and-white binary image using an OTU to eliminate background noise;

[0064] Tilt correction: The tilt angle of the text line is detected by Hough transform (e.g., 2.5°), and rotation correction is performed by applying affine transformation.

[0065] (3) Reprojection contour analysis

[0066] Vertical projection: Calculate the text pixel density of each column along the vertical direction of the image to generate a projection curve, such as... Figure 2 As shown.

[0067] in, Figure 2 The Mongolian translation of the record reads: "Although he was wounded six times in the war, but..."

[0068] Gaussian filtering: Uses a Gaussian kernel (σ = 3, kernel size 5 × 1) to smooth the projection curve and eliminate local noise;

[0069] Extreme point detection: Locate local minimum points (valleys) on a smooth curve to serve as the dividing boundaries between text lines.

[0070] (4) Cutting and post-processing

[0071] Text line extraction: The image is segmented based on the minimum points to generate a single-line text region (size: 32×1000 pixels);

[0072] Edge smoothing: Morphological closing operations (3×3 rectangular kernel) are performed on the segmented image to fill in small breaks. The output is a set of segmented text images (e.g., ...). Figure 1 (As shown).

[0073] Step 12, TPS correction: Predict the reference point to generate a regular rectangular text image.

[0074] In one text recognition embodiment, this step specifically includes:

[0075] (1) Input a single-line text image (which may be curved or tilted).

[0076] (2) Benchmark point prediction

[0077] Localization network: Employs a lightweight CNN (4 convolutional layers + fully connected layers), outputting K = 20 reference point coordinates (e.g., Figure 2 (Left side);

[0078] Normalization: The coordinate range is restricted to [-1, 1] using the tanh activation function. Mesh generation and transformation;

[0079] Baseline point setting: 20 fixed points are evenly distributed along the upper and lower edges of the calibrated image (e.g., ...). Figure 2 (Right image);

[0080] TPS parameter calculation: Based on the reference point and the base-reference point, the transformation matrix is ​​generated through differentiable matrix operations (such as formulas 1 and 2).

[0081]

[0082] (3) Bilinear interpolation sampling

[0083] Grid mapping: Mapping each pixel of the corrected image to the coordinates of the original image (Formula 3);

[0084]

[0085] Interpolation calculation: The weighted average of four adjacent pixels in the original image is used to generate a smooth, regular rectangular image (size: 32×1000 pixels).

[0086] (4) Output the corrected regular image (e.g.) Figure 2 ).

[0087] Step 13, Text line recognition: Decode the entire line of text based on the ResNet-BiLSTM-attention model.

[0088] In one text recognition embodiment, this step specifically includes:

[0089] (1) Model selection

[0090] Text line level: The ResNet-BiLSTM-attention model from the literature is used to achieve end-to-end text line recognition.

[0091] (1) Feature extraction

[0092] Global features: ResNet extracts the overall features of the text line image (such as layout and curvature).

[0093] (3) Decoding output

[0094] Text line recognition: Decoding the feature sequence into a complete line of text (e.g., Chinese meaning: Mongol Khan.

[0095] Step 14, word recognition: Perform character spacing analysis or semantic segmentation on the text line results, segment independent words, and perform secondary verification of easily confused characters.

[0096] In one word recognition embodiment, this step specifically includes:

[0097] (1) Model selection

[0098] Word-level: Based on an improved ResNet-BiLSTM architecture, optimizations are made for local word features (such as Mongolian concatenated characters).

[0099] (2) Feature extraction

[0100] Sequence modeling: BiLSTM captures long-range dependencies between characters within a text line (such as Mongolian glue grammar).

[0101] (3) Decoding output

[0102] Word segmentation and recognition: Segmenting words based on character spacing or semantic segmentation (e.g., Chinese meaning: Mongolia Chinese meaning: Khan; a character easily confused in lightweight CNN secondary verification.

[0103] Step 15, Feature Extraction and Decoding: Output character sequences using the ResNet-BiLSTM-Attention model.

[0104] In one text recognition embodiment, this step specifically includes:

[0105] (1) Input the corrected text image

[0106] (2) ResNet Feature Extraction

[0107] Backbone network: ResNet-34, BiLSTM hidden units 256, attention decoder beamwidth k=5, maximum character length 250.

[0108] Feature compression: The feature map is compressed into a 1D sequence (length 251, dimension 512) through 1×1 convolution.

[0109] (3) BiLSTM sequence modeling

[0110] Network structure: Two-layer BiLSTM (256 hidden units per layer), outputting context-aware feature sequences;

[0111] Feature concatenation: The hidden states of the forward and backward LSTMs are concatenated into a 512-dimensional vector.

[0112] (4) Attention Decoding

[0113] Attention mechanism: Additive attention is used, with dynamically weighted BiLSTM output;

[0114] Implicit language model: The decoder is a one-way LSTM (256 units), which combines the predicted character from the previous time step (such as Mongolian word roots) to generate the probability distribution of the current character;

[0115] Beam search: Set the beamwidth k=5 to retain the candidate sequence with the highest cumulative probability. Loss function and training.

[0116] Loss function: Cross-entropy loss (Equation 4.25);

[0117] Optimizer: Adam (learning rate 0.001, batch size 32);

[0118] Maximum character length: 250 characters. Output recognized text (e.g., Mongolian script transcribed into a Latin character sequence).

[0119] This invention employs thin-plate spline interpolation to correct curved / tilted text. The thin-plate spline transformation layer utilizes differentiable mesh generation technology to predict the reference points of the input image through differentiable matrix operations. Based on these reference points and a fixed base-reference point, TPS transformation parameters are calculated to generate a differentiable mesh. Interpolation is then used to generate a corrected, regular text image, achieving precise image transformation. This differentiable mesh defines the mapping relationship from the original curved / tilted state to the corrected state and can transmit gradients during backpropagation in the neural network, optimizing the geometric correction process. By combining differentiable mesh generation with bilinear interpolation, unsupervised text image regularization can be achieved, generating standard rectangular text regions and providing standardized input for subsequent processing.

[0120] The implementation of the thin plate spline conversion layer of the present invention can be specifically described by the following steps:

[0121] Step 121: Predict K reference points of the input image using a localization network;

[0122] Step 122: Using a mesh generator, based on the reference point and the fixed base-reference point, the TPS transformation parameters are calculated through differentiable matrix operations to generate a differentiable mesh;

[0123] Step 123: Based on the differentiable grid, the sampler is used to perform pixel mapping and weighted average calculation through bilinear interpolation to generate a corrected regular text image. Furthermore, bilinear interpolation can calculate the gradient during backpropagation, thereby optimizing the thin plate spline transformation layer in neural network training.

[0124] The attention decoding layer of this invention employs a beam search strategy to generate character sequences through the following steps:

[0125] Step 151: Calculate the weighted sum of the semantic vectors as the encoder's hidden states;

[0126] Step 152: Combine the predicted character embedding from the previous time step and update the decoding state using LSTM;

[0127] Step 153: Output the character probability distribution at the current time step.

[0128] Step 2: Label noise detection based on dynamic curriculum learning.

[0129] This step utilizes the MSL-MentorNet label noise detection module. Based on the text recognition results, it analyzes the loss sequence of the input image during consecutive training cycles, dynamically allocates sample weights to detect and filter noisy labels, and feeds the filtered data back to the text recognition module to optimize model training. Its main components include:

[0130] Temporal modeling of loss sequences across multiple time steps: A BiLSTM network is used to analyze the loss variation patterns of samples during continuous training cycles. The BiLSTM network receives the loss sequence, label type, and training progress information as input. Through bidirectional analysis of the loss sequence over time, it learns the characteristics of loss variation for different samples. For noisy labeled samples (such as IDN and RCN in data augmentation), their loss typically exhibits abnormal fluctuations or does not conform to the normal loss variation patterns of samples during continuous training cycles and multiple time steps; while the loss variation of clean samples is relatively stable and conforms to the expected pattern. Based on learning these loss variation patterns, the BiLSTM network outputs sample weights reflecting the probability that a sample is a noisy label. The higher the weight, the more likely the sample is to be a clean sample, thus distinguishing between noisy and clean samples.

[0131] Linear scheduling sample selection strategy: dynamically adjust the amount of training data, initially filter high-noise samples, and gradually introduce low-noise data to balance the robustness and generalization ability of the model.

[0132] The main content of step 2 above can be described as follows:

[0133] Multi-timestep loss sequence: The loss value of each input image in consecutive training cycles is recorded to construct a loss sequence. By analyzing the fluctuation characteristics and outliers of the loss sequence, samples that may have noisy labels are initially detected, providing a basis for subsequent processing.

[0134] Dynamic course learning: This process analyzes the loss sequence, dynamically allocates sample weights, and prioritizes training with low-noise samples. It is implemented using a BiLSTM-based MentorNet network. Specifically, the input includes the loss sequence, label type, and training progress information, while the output is sample weights. These weights reflect the probability that a sample is a noisy sample, enabling the detection and differentiation of noisy labeled samples.

[0135] Linear scheduling mechanism: The amount of training data is gradually increased according to the noise rate to balance noise filtering and model generalization. Specifically, the proportion of input images participating in training is dynamically adjusted based on the preset noise rate and the sample weights output by the BiLSTM discriminator network to optimize the training data.

[0136] Accordingly, the MSL-MentorNet label noise detection module of this invention mainly includes a loss sequence generation unit, a BiLSTM discriminant network, and a linear scheduling unit. These respectively implement the multi-timestep loss sequence, dynamic course learning, and linear scheduling mechanism mentioned above. The MSL-MentorNet label noise detection module achieves label noise detection functionality through the collaborative efforts of the loss sequence generation unit, BiLSTM discriminant network, and linear scheduling unit. The loss sequence generation unit is the basic component of MSL-MentorNet for obtaining sample training loss information, providing data for subsequent noise detection; the BiLSTM discriminant network is the core part of MSL-MentorNet that uses information such as loss sequences to discriminate noisy samples; the linear scheduling unit is the execution unit of MSL-MentorNet that dynamically manages training samples based on detection results, optimizing training data by adjusting the sample ratio.

[0137] Therefore, this invention distinguishes between noise labels and hard samples by analyzing the temporal changes of the loss sequence, avoiding the limitations of single-step thresholding. By using temporal modeling networks such as BiLSTM to analyze the loss sequence, it can effectively capture these pattern differences and avoid misclassifying hard samples as noise labels and erroneously discarding them when judging the threshold in single-step, thereby improving the accuracy and robustness of model training. Furthermore, this method can be extended to general datasets such as MNIST and supports robust training with noise rates of 10%-40%.

[0138] The method for constructing the loss sequence in this invention is as follows: Let the time step in the training process be t, t = 1, 2, ..., T, where T is the total number of time steps, and the number of consecutive training cycles be n, n = 1, 2, ..., N, where N is the total number of training cycles. For the i-th input image I... i :

[0139] At each time step within each training cycle, each image is input into the text recognition module and trained against the MSL-MentorNet label noise detection module. During each training iteration, the loss value is calculated based on the model's prediction results and the true labels, typically using common loss calculation methods such as the cross-entropy loss function. Where C is the number of categories, y i,j Let j be the value of the j-th class in the real label. Predict the probability of class j for the model;

[0140] During consecutive training cycles, the loss value is recorded sequentially at time steps to construct the loss sequence S. i ={L i,1,1 ,L i,1,2 ,...,L i,1,T ,L i,2,1 ,L i,2,2 ,...,L i,2,T ,...,Li,N,1 ,L i,N,2 ,...,L i,N,T This is to illustrate the loss change for each image under multiple time steps and continuous training cycles, where i = 1, 2, ..., M, and M is the total number of input images.

[0141] Based on this, the tag noise detection process of the MSL-MentorNet tag noise detection module of the present invention can be described as follows:

[0142] Step 21, coarse alignment preprocessing: Before data synthesis, coarse alignment processing is performed on the clean subset.

[0143] In one embodiment of tag noise detection, this step specifically includes:

[0144] (1) Coarse alignment preprocessing method

[0145] Before data synthesis, clean subsets are coarsely aligned (e.g., based on feature clustering or distribution similarity measures) to divide the samples into well-aligned subsets and poorly aligned subsets.

[0146] Well-aligned subsets: reserved for subsequent noise injection (IDN / RCN).

[0147] Poorly aligned subsets: temporarily store or remove them directly to avoid introducing potential noise interference.

[0148] (2) Advantages of coarse alignment preprocessing

[0149] This reduces the distribution offset of injected noise at the source, improving the rationality of subsequent noise synthesis. It also avoids repeatedly dealing with alignment issues in later steps, simplifying the process complexity.

[0150] Step 22, Data Synthesis: Inject IDN / RCN noise into a clean subset to generate training data.

[0151] In one embodiment of tag noise detection, this step specifically includes:

[0152] (1) Data synthesis (noise injection and training set construction)

[0153] Input a clean subset.

[0154] (2) Noise injection method

[0155] IDN:

[0156] Generation process: Inference is performed on a clean subset using a pre-trained TRBA model to obtain the model's predicted probability distribution for each sample; the incorrect category (non-true label) with the highest predicted probability is labeled to generate incorrect labels; Example: word (Chinese meaning: Mongolia) was mistakenly labeled as a model that is easily confused with other models. (Chinese meaning: Mongolian, visually similar character variant); Select the label with the highest predicted probability but incorrect, and inject it at a rate of 5%.

[0157] RCN:

[0158] Generation process: Randomly select 5% of the samples and replace their labels with any other arbitrary category (uniform distribution); Example: words (Chinese meaning: Sun) was randomly replaced with (meaningless combination); randomly replace labels with other categories, injection ratio 5%, total noise rate 10%.

[0159] (3) Output noise training set.

[0160] Step 23, MentorNet pre-training: Train the BiLSTM network with the loss sequence as input and binary weights as output.

[0161] In one embodiment of tag noise detection, this step specifically includes:

[0162] (1) MentorNet pre-training (BiLSTM discriminant network construction)

[0163] Input the loss sequence and label information of the samples in the noisy training set.

[0164] (2) Loss sequence generation

[0165] StudentNet pre-training: Train the initial TRBA model on a clean subset to generate basic feature extraction capabilities;

[0166] Loss Recording: During training on a noisy dataset, the cross-entropy loss value for each sample is recorded for 10 consecutive training epochs, constructing a temporal loss sequence L = {L1, L2, ..., L...} 10}

[0167] (3) BiLSTM network structure

[0168] Input features: Standardized loss sequence (10-dimensional, range [0,1]); Label category (185-dimensional one-hot encoding); Training progress (percentage of the current epoch out of the total epochs, integer encoding 0-99);

[0169] Network architecture: Bidirectional LSTM layer (64 hidden units, processing time-series loss sequences), fully connected layer (20 units, ReLU activation), output layer (1 unit, Sigmoid activation, output sample weights w∈[0,1]);

[0170] Loss function: Mean Squared Error (MSE), target weight is 1 (clean sample) or 0 (noisy sample).

[0171] (1) Training process

[0172] Optimizer: Adam (learning rate 0.001, batch size 128);

[0173] Training data: 50% of the samples in the noisy dataset were used to train MentorNet, and the remaining 50% were used for validation;

[0174] Performance metrics: Noise detection accuracy (the proportion of correctly classified noise samples).

[0175] (2) Output

[0176] The pre-trained MentorNet model can dynamically output sample weights.

[0177] Step 24, Dynamic Sample Selection: Based on the weight threshold output by MentorNet, filter noisy samples and gradually increase the amount of training data.

[0178] In one embodiment of tag noise monitoring, this step specifically includes:

[0179] (1) Initial stage of dynamic sample selection

[0180] Strategy: For the first 5 epochs, only samples with weight w > 0.99 (high-confidence clean samples) are selected for training;

[0181] Objective: To avoid the interference of noisy samples on the initial learning of the model.

[0182] (2) Linear scheduling phase

[0183] Strategy: Starting from the 6th epoch, gradually increase the sample retention ratio according to the formula: retention ratio = min(1-ε, 0.5+0.05×current epoch), where ε = 10%, ε is the noise rate, and the total number of epochs is 30.

[0184] Operation: In each epoch, samples are selected from high to low weight until the retention ratio is reached.

[0185] (3) Stable phase

[0186] Strategy: Retain 90% of the samples in the last 10 epochs (corresponding to a noise rate of 10%); Objective: To ensure that the model fully learns all potential clean samples and improves generalization ability.

[0187] The method can be extended to multilingual scenarios (such as Chinese and Arabic) by replacing dictionary resources and character sets to adapt to different language characteristics.

[0188] Refer again Figure 1 ,exist Figure 1 In the specific embodiment shown, the complete steps of the present invention are described as follows:

[0189] Step 1: TPS Transformation Layer, inputting the original text image (which may be curved or tilted); Reference Point Prediction: Predicting K=20 reference point coordinates using a lightweight CNN (4 convolutional layers + fully connected layers). Figure 1 Left figure); Mesh generation and sampling: based on reference points and fixed base-reference points ( Figure 1 (Right figure) Generate TPS transform parameters, and generate a corrected regular rectangular image through bilinear interpolation (solid line represents forward propagation, dashed line represents attention weight feedback).

[0190] The Mongolian text in the input image translates as: "Daxi, an elderly man from a military family in Bairin Left Banner, donated a cartload of salt and two gazelles to the front lines. He realized that drought relief required everyone's labor, so after autumn he set traps and caught a wolf, two foxes, two gazelles, two badgers, two bears, and a tiger, solving the problem of feeding his family of three. Although he is from a military family and is already 60 years old and unable to do heavy work, he worked even harder to avoid becoming a burden on others and donated two gazelles to the front lines, earning everyone's respect. Other military families who saw this followed suit."

[0191] Step 2: ResNet feature extraction, backbone network: ResNet-34, output 256-dimensional feature map (size 1×251×256); feature compression: compress the feature map into a 1D sequence (length 251, dimension 512) through 1×1 convolution.

[0192] Step 3: BiLSTM sequence modeling, network structure: two-layer BiLSTM (256 hidden units per layer), output context-aware feature sequence (forward and backward hidden states concatenated into a 512-dimensional vector).

[0193] Step 4: Attention Decoder; Attention Mechanism: Additive attention dynamically weighted BiLSTM output; Decoding Process: A unidirectional LSTM (256 units) is used to generate the current character probability distribution based on the predicted character from the previous time step, and beam search (k=5) is employed to optimize the output sequence. Technical Results: TPS correction improves the recognition rate of curved characters by 9.53%; the synergistic effect of BiLSTM and the attention model achieves an accuracy of 85.65% in recognizing long text lines.

[0194] Figure 3 The BiLSTM network structure and training process for MSL-MentorNet are as follows:

[0195] Step 1: Network Structure. Input: Loss sequence (standardized loss values ​​for 10 consecutive epochs); Label category (185-dimensional one-hot encoding); Training progress (percentage of the current epoch out of the total epochs, integer encoding 0-99). BiLSTM layer: 64 hidden units, processing the temporal loss sequence; Fully connected layer: 20 units (ReLU activation); Output layer: 1 unit (Sigmoid activation), output sample weights w∈[0,1].

[0196] Step 2: Training Process. Pre-training phase: Train MentorNet on a noisy dataset, aiming to minimize the mean squared error (MSE) between weights and true labels. Dynamic sample selection: During the warm-up period (first 5 epochs), only samples with w > 0.99 are selected. In the linear scheduling phase, the data volume is increased according to the formula: retention ratio = min(0.9, 0.5 + 0.05 × current epoch). In the stabilization phase, 90% of the samples are retained. Technical results: At a noise rate of 10%, the noise detection accuracy reaches 89.70%; the training time is controlled within 15.1 hours, which is more efficient than traditional methods.

[0197] The ablation experiments for text line and word recognition were compared. The baseline model was CRNN (VGG+BiLSTM+CTC), as shown in Table 1. The ablation module was used to remove or replace TRBA components one by one (e.g., remove TPS, replace ResNet with VGG, disable BiLSTM or attention decoder). The key results of the experiments are shown in Table 2.

[0198] Table 1 Comparison of ablation experiment results for text line and word recognition

[0199]

[0200] Table 2 Ablation Experiment Results

[0201]

[0202] It can be seen that TPS correction has a significant impact on noise detection accuracy (a decrease of 4.5%), while BiLSTM contributes a 2.2% improvement.

[0203] Figure 4 The steps to obtain the accuracy curve for tag noise detection are as follows:

[0204] Step 1, Experimental setup: Dataset: MNIST handwritten digits (40% IDN noise);

[0205] Comparison methods: Standard (no filtering), MentorNet, SPL, MSL-MentorNet;

[0206] Evaluation metric: Test set accuracy as a function of training epochs.

[0207] Step 2: Curve trend analysis. Standard: Accuracy significantly decreased after the 20th epoch due to overfitting noise (78.83% → 75.20%); MentorNet: Initially improved rapidly, then stabilized at 81.64%; MSL-MentorNet: Continuously increased, ultimately achieving an accuracy of 82.42%, with minimal fluctuation. Key conclusions: MSL-MentorNet, through temporal loss modeling, delays noise overfitting and maintains a stable upward trend; it still outperforms other methods under 40% high noise, validating the robustness of temporal discrimination.

[0208] Figure 5 To enhance the Test Accuracy-epoch test curve on the dataset, the following steps are taken:

[0209] Step 1, Initialization: Initialize the parameters of each model, such as weights. Different models may have different initialization methods; for example, neural networks often use random initialization or initialization based on pre-trained weights.

[0210] Step two, iterative training: In each epoch (training round), the model performs a complete traversal of the training set data. Training data is read into the model in mini-batch mode, the error between the predicted result and the true label (such as cross-entropy loss) is calculated, and the model parameters are updated using the backpropagation algorithm to minimize the loss value. For example, in gradient descent-based optimization algorithms, the model weights are adjusted based on the calculated gradient.

[0211] Step 3, Testing in Each Round: After each epoch, the model is tested using the test set data. The test data is input into the model to obtain prediction results, and evaluation metrics such as accuracy are calculated based on the prediction results and the true labels. For example, for a text recognition task, the accuracy is calculated as the proportion of correctly recognized text out of the total number of test texts.

[0212] Step 4: Record the results: Record the test accuracy of different models in each epoch to draw a line graph as shown in the figure, which will intuitively show how the accuracy of different models changes with the epoch during training, so as to compare the training effect and convergence speed of each model.

[0213] The results show that on the MMT-TR augmented dataset, the MSL-MentorNet achieves an average recognition rate improvement of 1.87% compared to the baseline model, and maintains an accuracy of 82.42% on the MNIST dataset with 40% IDN noise. The TRBA framework and the MSL-MentorNet module achieve synergistic optimization through a cascaded design: TRBA provides high-precision text recognition results as input for noise detection, and MSL-MentorNet dynamically filters noise labels and feeds them back to the TRBA training process, forming a closed-loop optimization system. High-precision text recognition is achieved through the TRBA framework, and the dynamic filtering of noise labels by MSL-MentorNet improves the model's robustness on low-quality data. The system includes the following core modules: TRBA augmented text recognition framework: It corrects curved / tilted text images through a Thin Plate Spline (TPS) transformation layer, extracts deep visual features using a ResNet-BiLSTM-attention model to capture long-range contextual dependencies, and utilizes an implicit language model to improve the recognition ability of blurred and missing characters. Experiments show that TRBA achieves a text line spelling accuracy of 85.65% and a character shape accuracy of 89.71% on the Mongolian movable type dataset.

Claims

1. A label noise detection method based on multi-timestep loss sequences, characterized in that, Includes the following steps: Step 1: Use the text recognition module to recognize text from the input image; Step 2: Using the MSL-MentorNet label noise detection module, based on the text recognition results, analyze the loss sequence of the input image in continuous training cycles, dynamically allocate sample weights to detect and filter noise labels, and feed the filtered data back to the text recognition module to optimize model training. Before performing MSL-MentorNet detection in step 2, the following steps are also included: using feature clustering or distribution similarity measurement to perform coarse alignment processing on the initial clean subset, dividing the samples into well-aligned subsets and poorly aligned subsets. The well-aligned subsets are retained for subsequent noise injection, while the poorly aligned subsets are temporarily stored or directly removed to avoid introducing potential noise interference. The MSL-MentorNet tag noise detection module includes: The loss sequence generation unit records the loss value of each input image in consecutive training cycles and constructs a loss sequence. The BiLSTM discriminative network takes the loss sequence, label type, and training progress information as input and outputs sample weights. The weights reflect the probability that a sample is a noise sample, thus enabling the detection and differentiation of noise-labeled samples. The linear scheduling unit dynamically adjusts the proportion of input images participating in training based on the preset noise rate and the sample weights output by the BiLSTM discriminant network, thereby optimizing the training data. The linear scheduling unit adjusts the amount of data participating in training according to the following rules: Step 1: In the initial stage, only high-weight samples are selected for training; the high-weight samples refer to the samples that are judged to be more reliable and more likely to be correctly labeled after processing by the BiLSTM discriminant network in the MSL-MentorNet label noise detection module. Step 2: The sample retention rate is linearly increased with the training cycle until it reaches the preset 1-ε, where ε is the noise rate.

2. The label noise detection method based on multi-timestep loss sequence according to claim 1, characterized in that, The text recognition module performs geometric correction, visual feature extraction, and sequence decoding on the input image to obtain the text recognition result. The geometric correction is achieved by correcting curved and tilted text through a thin plate spline transformation layer, which utilizes a differentiable mesh to realize image transformation.

3. The label noise detection method based on multi-timestep loss sequence according to claim 1 or 2, characterized in that, The text recognition module includes: The thin plate spline transformation layer predicts the reference points of the input image through differentiable matrix operations, calculates the TPS transformation parameters based on the reference points and fixed base-reference points, generates a differentiable grid, and generates a corrected regular text image through interpolation. The ResNet feature extraction layer extracts multi-scale visual features from the corrected regular text image; A bidirectional LSTM layer is used to perform contextual modeling on the feature sequence; The attention decoding layer, combined with the implicit language model, decodes the feature sequence into a character sequence, which is the text recognition result.

4. The label noise detection method based on multi-timestep loss sequence according to claim 3, characterized in that, The implementation of the thin plate spline conversion layer includes: Step 1: Predict K reference points of the input image using a localization network; Step 2: Using a mesh generator, based on the reference point and the fixed base-reference point, calculate the TPS transformation parameters through differentiable matrix operations to generate a differentiable mesh; Step 3: Based on the differentiable grid, the sampler is used to perform pixel mapping and weighted average calculation through bilinear interpolation to generate the corrected regular text image. The bilinear interpolation performs gradient calculation during backpropagation to optimize the thin plate spline transformation layer in the neural network training.

5. The label noise detection method based on multi-timestep loss sequence according to claim 3, characterized in that, The attention decoding layer employs a beam search strategy to generate character sequences through the following steps: Step 1: Calculate the weighted sum of the semantic vectors as the encoder's hidden states; Step 2: Combine the predicted character embedding from the previous time step and update the decoding state using LSTM; Step 3: Output the character probability distribution at the current time step.

6. The label noise detection method based on multi-timestep loss sequence according to claim 1, characterized in that, The loss sequence generation unit constructs the loss sequence as follows: Let the time step in the training process be t, t=1,2,...,T, where T is the total number of time steps, and the number of consecutive training cycles be n, n=1,2,...,N, where N is the total number of training cycles. For the th... i Input image : At each time step within each training cycle, each image is input into the text recognition module and trained with the MSL-MentorNet label noise detection module. During each training iteration, the loss value is calculated based on the model's prediction results and the true labels, using the cross-entropy loss function. ,in For the number of categories, The first in the real label The values ​​that a class can take. For the model to predict the first The probability of a class; Record the loss value sequentially at time steps during consecutive training cycles to construct a loss sequence. This is to reflect the loss change for each image under multiple time steps and continuous training cycles, where i=1,2,...,M, and M is the total number of input images.

7. The label noise detection method based on multi-timestep loss sequence according to claim 1, characterized in that, The BiLSTM discriminant network is trained through the following process: Step 1: Synthesize IDN and RCN noisy data on a clean subset; the clean subset refers to a portion of data selected from the original dataset that is accurately labeled and free of noise labels. Step 2: Using the loss sequence, label type, and percentage of training cycles as inputs, and binary weights as outputs, minimize the mean squared error; Step 3: Update network parameters using the Adam optimizer.

8. The label noise detection method based on multi-timestep loss sequence according to claim 7, characterized in that, The method for synthesizing IDN noise data on a clean subset is as follows: 1) Set the IDN noise rate ρIDN, which represents the proportion of samples in the clean subset to have feature-related noise added; 2) Randomly select a number of samples nIDN=ρIDN×P from the clean subset, where P is the total number of samples in the clean subset; 3) For each selected sample, analyze its characteristics; 4) Modify the labels according to the correlation between sample features and other category features to generate noise labels related to sample features, thus completing the synthesis of IDN noise data; The method for synthesizing RCN noise data on a clean subset is as follows: 1) Set the RCN noise rate ρRCN, which represents the proportion of samples to be added with random class noise in the clean subset; 2) Randomly select a sample of size nRCN = ρRCN × P from the clean subset; 3) For each selected sample, randomly replace its true label with a label from another category in the dataset; Complete the synthesis of RCN noise data.

Images