Instrument optical character recognition method based on multi-scale space perception and task collaborative learning

Abstract

The invention provides an instrument optical character recognition method based on multi-scale spatial perception and task collaborative learning. The method comprises the following steps: S1, performing data enhancement and batch size unification preprocessing on pictures in a data set; s2, sending the preprocessed picture into an image feature extractor for multi-scale visual feature extraction; s3, extracting character position information according to the multi-scale visual features, and embedding position information for the multi-scale visual features; s4, inputting the multi-scale visual features into a Transform encoder to learn global features to obtain a shared coding feature FSEF; s5, loss supervision is carried out, and shared coding feature representation is optimized; s6, finally combining multi-scale feature learning loss, pixel segmentation loss, alignment irrelevant loss and decoding prediction loss to perform end-to-end optimization of the model; end-to-end optimization is performed through an MSCL-OCR architecture weighted joint loss function, coding features are fully utilized and optimized, and recognition of multi-scale, variable-length and complex layout characters of industrial instruments and meters is realized.

Description

Technical Field

[0001] This invention relates to the field of optical character recognition technology for instruments and meters, and in particular to an optical character recognition method for instruments and meters that combines multi-scale spatial perception and task collaborative learning. Background Technology

[0002] Optical Character Recognition (OCR) technology for instruments and meters has made significant progress driven by deep learning. Currently, it mainly extracts visual features through CNNs and then decodes them into character sequences by RNNs to achieve OCR recognition of instruments and meters in industrial scenarios. With the great success of the Transformer architecture in the field of natural language processing, Transformer has shown significant advantages in processing text with complex two-dimensional layouts due to its powerful sequence modeling capabilities and global dependency capture capabilities. The introduction of Graph Neural Networks (GNNs) has further enhanced the ability to model non-Euclidean structures in text images, enabling modern instrument and meter OCR systems to handle text recognition tasks in complex scenarios more effectively.

[0003] However, OCR recognition of instruments and meters faces a series of severe challenges, including multi-scale variations, variable length characteristics, and uneven character spacing of instrument and meter characters. These challenges are coupled with each other, forming a complex intertwined effect. For the challenge of multi-scale variations, the limitation of the fixed receptive field design of CNN-RNN models is that they cannot adaptively extract multi-scale features of instrument and meter characters, affecting the model's ability to perceive information at different scales. For the challenge of spatial layout complexity, fixed position encoding or learnable position encoding cannot adapt to complex spatial layout changes, resulting in feature alignment bias when the model processes text with irregular spacing. For the challenge of recognizing variable-length sequences of instrument and meter characters, the use of CTC and Attention single-branch or a combination of both dual-branch structures lacks a multi-angle supervision mechanism to fully optimize the shared encoded feature representation. Summary of the Invention

[0004] The purpose of this invention is to solve the technical problems existing in the optical character recognition of instruments and meters, and in particular, to provide a method for optical character recognition of instruments and meters that combines multi-scale spatial perception and task collaborative learning.

[0005] To achieve the above-mentioned objectives of this invention, this invention provides a method for optical character recognition of instruments and meters based on multi-scale spatial perception and task collaborative learning, comprising the following steps:

[0006] S1: Perform data augmentation and batch size standardization preprocessing on the images in the dataset;

[0007] S2: The preprocessed image is fed into the image feature extractor for multi-scale visual feature extraction;

[0008] S3: Extract character position information based on multi-scale visual features and embed the position information into the multi-scale visual features;

[0009] S4: Input multi-scale visual features containing location information into the Transformer encoder to learn global features and obtain shared encoded features F. SEF ;

[0010] S5: The F-axis is composed of segmentation (segment), text alignment CTC, and decoder attention. SEF Perform loss supervision and optimize the shared coding feature representation;

[0011] S6: Finally, the multi-scale feature learning loss, pixel segmentation loss, alignment irrelevant loss, and decoding prediction loss are combined to optimize the model end-to-end until the model fits and reaches the optimal result.

[0012] Preferably, the data augmentation and batch size unification preprocessing includes:

[0013] A 1x1 convolutional layer is used to process each original feature map {P} i The number of channels in the model is uniformly projected onto the hidden dimension d of the model. model The original feature map {P} is transformed using bilinear interpolation. i The spatial resolution of the target is uniformly adjusted to match the target size, resulting in a preprocessed target feature map. The specific calculation process is as follows:

[0014] Let the original feature map {P} i The size of} is (H) i W i The target size is (H). t arget,W t If the value is `arget`, then the scaling factor is: `scale`. h =H t arget / H i scale w =W t arget / W i For any position (x′, y′) in the target feature map, its original coordinates are: x = x′ / scale w y = y′ / scale h Bilinear interpolation uses a weighted average of the four nearest neighbor pixel values. First, the coordinates of the four nearest neighbor pixels are determined, and the calculation formula is as follows:

[0015] Where x1 represents the value of x rounded down, x2 represents the value of x1 plus 1, y1 represents the value of y rounded down, and y2 represents the value of y1 plus 1. This indicates a round-down operation;

[0016] The formula for calculating the interpolated pixel value is as follows:

[0017]

[0018] Among them, P i (x1, y1), P i (x2, y1), P i (x1, y2) and P i (x2, y2) represent the pixel values ​​of the four nearest neighbors in the original feature map, P i ′(x′, y′) represents the interpolated pixel value at position (x′, y′) in the target feature map; this interpolation process is used to combine all the original feature maps {P i Adjust to the same spatial resolution (H t arget, W t target feature map of arget)

[0019] Furthermore, the multi-scale visual feature extraction includes:

[0020] For each aligned target feature map Extract its global context descriptor Where i equals an integer from 1 to N; context descriptors of all scales are concatenated dimensionally using the Concat function, input to a scale-attention multilayer perceptron, and learn to predict a normalized attention weight distribution α = α1, α2, ..., α N The calculation formula is as follows:

[0021] α=Softmax(SA_MLP(Concat(s1,s2,...,s N )));

[0022] Where Softmax() is the probability normalization function, SA_MLP is the scale-attention multilayer perceptron, Concat() represents the dimension concatenation function, and N represents the number of scales fused.

[0023] Finally, the fused target feature map F fused It is generated by dynamically weighting and summing feature maps at various scales, and its calculation formula is as follows:

[0024]

[0025] Where, α i Represents the weighting coefficients of the i-th scale feature map. This represents the feature map at the i-th scale.

[0026] Furthermore, S3 includes:

[0027] Content-adaptive 2D conditional location coding (CPE) is used to output a two-dimensional feature map F from the feature extractor. enc As input to the convolutional network, a 2D location-encoded map (PE) closely related to the current image content features is dynamically generated. cond ; and then with F enc Element-wise addition is performed to form the initial state F of the sequence representation that is finally fed into the Transformer encoder. enc′ The calculation formula is as follows:

[0028] F enc′ =F enc +PE cond ;

[0029] Where + is the element-wise addition symbol, F enc For the fused target feature map F fused PE cond This represents a 2D positional encoding diagram.

[0030] Furthermore, S4 includes:

[0031] The Transformer encoder represents the initial state F as a sequence. enc′ Convert to shared coding features H, W, and D represent height, width, and feature dimension, respectively, serving as the unified feature foundation for all downstream tasks. The TCLM module constructs a four-fold collaborative supervision framework, with scale-aware loss guiding optimal multi-scale fusion during feature extraction. The three parallel branches—segmentation, attention, and CTC—contribute to F from the perspectives of spatial structure, semantic content, and temporal alignment, respectively. SEF Provide differentiated monitoring signals.

[0032] Furthermore, S5 includes:

[0033] S5-1: Dynamically allocate loss weights based on sample scale categories to ensure differentiated attention is given to characters at different scales;

[0034] S5-2: To enhance the model's adaptability to multi-scale features, a scale-aware attention supervision mechanism is introduced;

[0035] S5-3: The three parallel branches work together to optimize the shared coding features through different supervision signals, forming a complementary learning mechanism.

[0036] Furthermore, the differentiated attention given to characters of different scales includes:

[0037] The dynamic weights of sample i in the t-th training round The calculation formula is:

[0038]

[0039] Where α = 0.8 and β = 1.5 are scaling factors, w base =1.0 is the base weight, τ high =0.95 and τ low =0.85 is the performance threshold. Cat represents the class performance of sample i. i Indicates the scale category of sample i. This indicates the category performance statistics for the previous round.

[0040] Furthermore, the scale-aware attention supervision mechanism includes:

[0041] Its supervisory losses The calculation formula is:

[0042]

[0043] Where, λ ent and λ div λ is the weighting coefficient that balances the different loss terms. ent =0.1, λ div =0.05; To monitor losses, For the divergence loss term, For entropy regularization, For diversity loss;

[0044] The supervised loss is composed of three complementary terms: The feature extractor learns a scale-adaptive attention pattern by being guided;

[0045] Divergence loss term: The main supervisory signal is used to force the prediction of attention weights. With the target distribution α target Alignment is achieved between elements, and the calculation formula is as follows:

[0046]

[0047] Where KL(||) represents Kullback-Leibler divergence alignment, α pred Let α represent the predicted attention weight vector. target Let α represent the target attention weight vector. target,i and α pred,i They represent α respectively target and α pred The i-th element in the vector;

[0048] Entropy regularization: To prevent overconfident attention distributions, we employ entropy regularization, which is calculated as follows:

[0049]

[0050] Diversity loss term: To encourage diversity of attention among batches, we minimize the deviation from a uniform distribution, which is calculated as follows:

[0051]

[0052] in, This represents the average attention level across batches. For a uniformly distributed vector, Let B represent the predicted attention weight vector for the b-th sample, where B represents the batch size. This represents the calculation of the square of the L2 norm.

[0053] Furthermore, the three parallel branches include:

[0054] The formula for calculating the feature representation branch that enables the encoder to learn temporally robust features is as follows:

[0055]

[0056] in, This indicates that the encoder learns temporally robust feature representations. Let P represent all possible aligned paths of the target sequence Y, where T represents the time step and P represents the time step. ctc (π t |F SEF ) represents the CTC probability at time step t, π t This represents the value of path π at time t;

[0057] The segmentation branch enhances spatial discrimination capabilities through pixel-level segmentation, particularly for precise character boundary localization. Its calculation formula is as follows:

[0058]

[0059] in, This indicates that the segmentation branch enhances spatial discriminative ability through pixel-level segmentation. To address focus loss due to class imbalance, λ is the U-shaped space regularization penalty term; u =1.0 indicates the weight of the penalty term;

[0060] To prevent the model from generating trivial solutions with all foreground or all background, the attention branch serves as the main recognition path, providing semantic-level supervision through autoregressive decoding. Its calculation formula is as follows:

[0061]

[0062] in, This represents the attention branch, where L is the length of the target sequence. This is the actual marker at position t. Let F represent all preceding labels, logP(|) represent the logarithm of the probability, and F SEF This indicates shared coding features.

[0063] Furthermore, S6 includes:

[0064] The multi-task collaboration mechanism achieves end-to-end optimization of the total value through a weighted joint loss function. The calculation formula is as follows:

[0065]

[0066] Where, λ att =1.0 represents the attention weight. Indicates the attention branch; λ ctc =0.2 represents the robust feature weight. This indicates that the encoder learns temporally robust feature representations; λ seg =0.2 represents the discrimination ability weight. Indicates branching; λ scale =0.2 is the weight of the supervised loss. This indicates a loss of oversight.

[0067] In summary, by utilizing the above technical solutions, the instrument optical character recognition method proposed in this invention, which combines multi-scale spatial perception and task collaborative learning, employs a multi-scale feature extraction method with shared encoder features as the core. It achieves more refined visual feature representation by introducing a multi-task collaborative supervision mechanism and performs end-to-end optimization through a weighted joint loss function. This fully utilizes and optimizes the encoded features to realize the recognition of multi-scale, variable-length, and complex layout characters of industrial instruments. Attached Figure Description

[0068] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0069] Figure 1 This is a diagram of the multi-scale convolutional backbone network architecture of the present invention.

[0070] Figure 2 This is the MSCL-OCR model architecture diagram of the present invention.

[0071] Figure 3 This is a schematic diagram of the process of this invention.

[0072] Figure 4 This is a comparison chart of scale perception attention weight allocation analysis. Detailed Implementation

[0073] The embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0074] To effectively capture text features of varying sizes and shapes in images of instruments and meters in industrial settings, a multi-scale convolutional backbone network was designed, such as... Figure 1 As shown, the design of this network architecture draws on the idea of ​​cross-stage local feature fusion in advanced visual models; the backbone network is composed of multiple multi-branch convolutional stage modules cascaded together, which can aggregate features in layers and gradually expand the receptive field.

[0075] The core of the multi-scale convolutional backbone network is the multi-kernel bottleneck module, which aims to enhance the network's ability to represent features of different character structures. Specifically, unlike the standard bottleneck structure, the multi-kernel bottleneck module contains multiple parallel convolutional branches. Each branch uses convolutional kernels of different sizes (e.g., 3×3 and 5×5). This multi-kernel parallel architecture enables the model to capture contextual information from multiple receptive fields simultaneously within a single module. Therefore, the network can more flexibly adapt to characters with different sizes, aspect ratios, and levels of detail.

[0076] We also designed a multi-scale spatial perception and collaborative learning framework for optical character recognition of instruments, abbreviated as MSCL-OCR, such as... Figure 2 As shown, MSCL-OCR employs a multi-scale feature extraction method with shared encoder features as its core, and introduces a three-branch collaborative learning paradigm of recognition-alignment-segmentation.

[0077] In addition, to ensure the training stability of the model under different batch sizes and data distributions, the MSCL-OCR model uses group normalization after the scale-aware attention module. Unlike batch normalization, group normalization is performed in groups along the channel dimension, so that its performance is not affected by the batch size. This is especially important for scenarios that use gradient accumulation or are trained on different hardware.

[0078] like Figure 3 The diagram shown illustrates the flowchart of an instrument optical character recognition method based on multi-scale spatial perception and task collaborative learning, including:

[0079] S1: Perform data augmentation and standardize batch size for images in the dataset.

[0080] Specifically, including:

[0081] S1-1. Employing a Feature Pyramid Network (FPN), which combines high-level semantic information with low-level spatial details through lateral connections and top-down paths, generates a series of original feature maps rich in contextual information and effectively fusing multi-scale features extracted from the backbone network for subsequent tasks. i}

[0082] S1-2, For each original feature map {P} in the Feature Pyramid Network (FPN) i This process is performed to achieve alignment in both channel and spatial dimensions.

[0083] Specifically, a 1x1 convolutional layer is used to process each original feature map {P} i The number of channels in the model is uniformly projected onto the hidden dimension d of the model. model The original feature map {P} is transformed using bilinear interpolation. i The spatial resolution of the target is uniformly adjusted to match the target size, resulting in a preprocessed target feature map. The specific calculation process is as follows:

[0084] Let the original feature map {P} i The size of} is (H) i W i The target size is (H). t arget, W t If the value is `arget`, then the scaling factor is: `scale`. h =H t arget / H i scale w =W t arget / W i For any position (x′, y′) in the target feature map, its original coordinates are: x = x′ / scale w y = y′ / scale h Bilinear interpolation uses a weighted average of the four nearest neighbor pixel values. First, the coordinates of the four nearest neighbor pixels are determined, and the calculation formula is as follows:

[0085]

[0086] Where x1 represents the value of x rounded down, x2 represents the value of x1 plus 1, y1 represents the value of y rounded down, and y2 represents the value of y1 plus 1. This indicates a round-down operation;

[0087] The formula for calculating the interpolated pixel value is as follows:

[0088] P i'(x', y')=(x²-x)(y²-y)P i (x1,y1)+(x-x1)(y2-y)P i (x2, y1) + (x2 - x)(y - y1)P i (x1,y2)+(x-x1)(y-y1)P i (x2, y2)

[0089] Among them, P i (x1, y1), P i (x2, y1), P i (x1, y2), P i (x2, y2) represents the pixel values ​​of the four nearest neighbors in the original feature map, P i ′(x′, y′) represents the interpolated pixel value at position (x′, y′) in the target feature map; this interpolation process is used to combine all the original feature maps {P i Adjust to the same spatial resolution (H t arget, W t target feature map of arget)

[0090] S2: The preprocessed image is fed into the image feature extractor for multi-scale visual feature extraction.

[0091] Specifically, including:

[0092] To adaptively extract the most discriminative information from the Feature Pyramid Network (FPN), a scale-aware attention fusion mechanism is proposed.

[0093] Specifically, for each aligned target feature map Extract its global context descriptor Where the integers can be from 1 to N; context descriptors at all scales are concatenated dimensionally using the Concat function, input to a scale-attention multilayer perceptron, and learn to predict a normalized attention weight distribution α = α1, α2, ..., α N The calculation formula is as follows:

[0094] α=Softmax(SA_MLP(Concat(s1,s2,…,s N )))

[0095] Where Softmax is the probability normalization function, SA_MLP() is the scale-attention multilayer perceptron, Concat() represents the dimension concatenation function, and N represents the number of scales fused.

[0096] Finally, the fused target feature map F fusedIt is generated by dynamically weighting and summing feature maps at various scales, and its calculation formula is as follows:

[0097]

[0098] Where, α i Represents the weighting coefficients of the i-th scale feature map. This represents the feature map at the i-th scale.

[0099] S3: Extract character position information based on multi-scale visual features and embed the position information into the multi-scale visual features;

[0100] Specifically, including:

[0101] To address the issues of variable target scale and uneven character spacing in instrument text under complex industrial scenarios, traditional fixed or simple learnable positional coding cannot provide optimal spatial priors.

[0102] Therefore, content-adaptive 2D conditional location coding (CPE) is adopted, the core of which is to output a two-dimensional feature map F from the feature extractor. enc As input to the convolutional network, a 2D location-encoded map (PE) closely related to the current image content features is dynamically generated. cond ; and then with F enc Element-wise addition is performed to form the initial state F of the sequence representation that is finally fed into the Transformer encoder. enc′ The calculation formula is as follows:

[0103] F enc' =F enc +PE cond

[0104] Where + is the element-wise addition symbol, F enc For the fused target feature map F used PE cond Represents a 2D positional coding diagram;

[0105] S4: Input multi-scale visual features containing location information into the Transformer encoder to learn global features and obtain shared encoded features F. SEF ;

[0106] Specifically, this paper addresses the significant challenges of recognizing variable-length character sequences in industrial instruments due to scale heterogeneity and sequence diversity. To solve this problem, the paper proposes a Task Collaborative Learning (TCLM) module, which optimizes the feature learning of the main task through multiple auxiliary tasks. TCLM employs a four-fold collaborative supervision mechanism to provide multi-view supervision signals for encoder features, thus addressing the complex challenges of variable-length character sequences in industrial instruments.

[0107] The Transformer encoder represents the initial state F as a sequence. enc′ Convert to shared coding features H, W, and D represent height, width, and feature dimension, respectively, serving as the unified feature foundation for all downstream tasks. The TCLM module constructs a four-fold collaborative supervision framework, with scale-aware loss guiding optimal multi-scale fusion during feature extraction. The three parallel branches—segmentation, attention, and CTC—contribute to F from the perspectives of spatial structure, semantic content, and temporal alignment, respectively. SEF Provide differentiated supervision signals; the multi-task collaborative mechanism uses gradient backpropagation to make the single encoded feature F SEF It can simultaneously meet the combined requirements of spatial positioning accuracy, semantic understanding depth, and temporal modeling robustness, achieving a superlinear synergistic gain effect.

[0108] S5: The F-axis is composed of segmentation (segment), text alignment CTC, and decoder attention. SEF Perform loss supervision and optimize the shared coding feature representation;

[0109] Specifically, including:

[0110] S5-1. Considering the impact of multi-scale characteristics on recognition difficulty, this paper implements a dynamic weight adjustment strategy based on scale category. This mechanism dynamically allocates loss weights according to the sample scale category, ensuring differentiated attention to characters at different scales. The dynamic weight of sample i in the t-th training round is... The calculation formula is:

[0111]

[0112] Where α = 0.8 and β = 1.5 are scaling factors, w base =1.0 is the base weight, τ high =0.95 and τ low =0.85 is the performance threshold. Cat represents the class performance of sample i. i Indicates the scale category of sample i. This represents the category performance statistics for the previous round; the system divides characters into four scale categories: micro_tiny, small_medium, medium_large, and large_gaint; i represents the size category, namely micro_tiny, small_medium, medium_large, and large_gaint;

[0113] Target attention pattern: target distribution α target Based on the following predefined scale categories:

[0114] ·micro_tiny: (P3-driven, capturing fine details);

[0115] ·small_medium: (P4 dominant, balanced characteristic representation);

[0116] •medium_large: (P5 tendency, coarse-grained structure);

[0117] ·large_giant: (P5 dominant, global context);

[0118] This strategy improves the model's ability to learn from difficult samples by adaptively adjusting the importance of samples during training by monitoring the recognition performance of categories at various scales in real time.

[0119] S5-2. To enhance the model's adaptability to multi-scale features, we introduce a scale-aware attention supervision mechanism, whose supervision loss... The calculation formula is:

[0120]

[0121] Where, λ ent and λ div λ is the weighting coefficient that balances the different loss terms. ent =0.1, λ div =0.05; To monitor losses, For the divergence loss term, For entropy regularization, For diversity loss;

[0122] The supervised loss is composed of three complementary terms: The feature extractor learns a scale-adaptive attention pattern by being guided;

[0123] Divergence loss term: The main supervisory signal is used to force the prediction of attention weights. With the target distribution α target Alignment is achieved between elements, and the calculation formula is as follows:

[0124]

[0125] Where KL(||) represents Kullback-Leibler divergence alignment, α pred Let α represent the predicted attention weight vector. target Let α represent the target attention weight vector. target,i and α pred,i They represent α respectively target and αpred The i-th element in the vector;

[0126] Entropy regularization: To prevent overconfident attention distributions, we employ entropy regularization, which is calculated as follows:

[0127]

[0128] Where, α pred,i α pred The i-th element in the vector;

[0129] Diversity loss term: To encourage diversity of attention among batches, we minimize the deviation from a uniform distribution, which is calculated as follows:

[0130]

[0131] in, This represents the average attention level across batches. For a uniformly distributed vector, Let B represent the predicted attention weight vector for the b-th sample, where B represents the batch size. This represents the calculation of the square of the L2 norm.

[0132] This triple-component supervised model effectively guides the adoption of optimal feature fusion strategies while maintaining appropriate flexibility under diverse input conditions.

[0133] S5-3. The three parallel branches collaboratively optimize shared coding features through different supervision signals, forming a complementary learning mechanism. The CTC branch provides alignment-independent temporal supervision, prompting the encoder to learn temporally robust feature representations. Its calculation formula is as follows:

[0134]

[0135] in, This indicates that the encoder learns temporally robust feature representations. Let P represent all possible aligned paths of the target sequence Y, where T represents the time step and P represents the time step. ctc (π t |F SEF ) represents the CTC probability at time step t, π t This represents the value of path π at time t.

[0136] The segmentation branch enhances spatial discrimination capabilities through pixel-level segmentation, particularly for precise character boundary localization. Its calculation formula is as follows:

[0137]

[0138] in, This indicates that the segmentation branch enhances spatial discriminative ability through pixel-level segmentation. To handle focus loss due to class imbalance, λ u =1.0 indicates the weight of the penalty term. The U-shaped space regularization penalty term prevents the model from generating trivial solutions with all foreground or all background; attention branch function. As the primary recognition path, semantic-level supervision is provided through autoregressive decoding, and its calculation formula is as follows:

[0139]

[0140] in, This represents the attention branch, where L is the length of the target sequence. This is the actual marker at position t. Let F represent all preceding labels, logP(|) represent the logarithm of the probability, and F SEF This indicates shared coding features.

[0141] S6. Finally, the multi-scale feature learning loss, pixel segmentation loss, alignment irrelevant loss and decoding prediction loss are combined to optimize the model end-to-end until the model fits and reaches the optimal result.

[0142] Specifically, including:

[0143] The multi-task collaboration mechanism achieves end-to-end optimization of the total value through a weighted joint loss function. The calculation formula is as follows:

[0144]

[0145] Where, λ att =1.0 represents the attention weight. Indicates the attention branch; λ ctc =0.2 represents the robust feature weight. This indicates that the encoder learns temporally robust feature representations; λ seg =0.2 represents the discrimination ability weight. Indicates branching; λ scale =0.2 is the weight of the supervised loss. This indicates a loss of oversight.

[0146] This weight allocation strategy ensures that the primary recognition task receives sufficient gradient signals, while the auxiliary task provides effective regularization constraints. Through multi-branch gradient collaboration and shared encoded feature representation learning, this multi-task architecture enables the encoder to simultaneously optimize temporal modeling, spatial localization, and semantic understanding tasks within a unified framework, thereby significantly enhancing the model's adaptability to scale diversity and sequence length variations in industrial scenarios. Compared to traditional single-supervised learning methods, the proposed scale-aware collaborative enhancement mechanism exhibits stronger generalization ability and recognition robustness when processing heterogeneous character sequences.

[0147] To further illustrate the effectiveness of this invention and comprehensively evaluate the performance of the proposed method, this paper selects five representative state-of-the-art (SOTA) text recognition methods as baselines for comparative experiments. These methods cover the technological development from CNN-RNN architecture to the latest Transformer architecture, specifically including:

[0148] 1. CRNN (2017) adopts the classic convolutional neural network-recurrent neural network-connection temporal classification architecture, uses VGG as a feature extractor, BiLSTM to capture sequence dependencies, and CTC for sequence alignment. It is a classic baseline method in the field of industrial recognition technology.

[0149] 2. RARE (2016) introduced a spatial transformation network (TPS) for geometric correction and combined it with ResNet feature extraction and attention mechanism for sequence recognition, which performed well in processing curved and deformed text.

[0150] 3. ASTER (2018) integrates BiLSTM and attention mechanism on the basis of RARE. Through the combined architecture of TPS+ResNet+BiLSTM+attention mechanism, it has achieved significant improvement in text recognition in complex scenes.

[0151] 4. TrOCR (2021) adopts the Transformer architecture, using a pre-trained ViT as the visual encoder and a text Transformer as the decoder, demonstrating the potential of end-to-end Transformer in OCR tasks.

[0152] 5. PARSeq (2022) is based on the permutation autoregressive mechanism, which unifies autoregressive, non-autoregressive and bidirectional contextual modeling, and represents the latest technological development in sequence recognition.

[0153] 2. Multi-scale variation challenge verification experiment

[0154] Addressing the core challenge of varying character scales in industrial instruments, this section designs comprehensive multi-scale validation experiments to systematically evaluate the effectiveness of the MSAF-Net multi-scale attention feature extraction network.

[0155] 2.1 Performance Comparison of Multi-Scale Character Recognition

[0156] To systematically evaluate the effectiveness of the proposed MSAF-Net in solving multi-scale variation problems, we conducted a comprehensive comparative experiment with representative baseline methods for four different character scale categories; Table 1 shows the quantitative performance analysis results.

[0157] Table 1: Performance Comparison of Multi-Scale Character Recognition Methods

[0158] method Micro-tiny (AC%)↑ Small-medium (AC%)↑ Medium-large (AC%)↑ Large-giant (AC%)↑ MSR (%) ↑ CRNN 95.89 90.52 84.02 96.73 0.8671 ASTER 97.44 95.58 94.19 98.52 0.9480 RARE 97.45 95.43 94.09 98.44 97.45 TrOCR 98.53 96.25 94.85 99.22 0.9546 SPRK-OCR 97.46 95.50 99.48 99.67 0.9633

[0159] Experimental results show that the proposed MSCL-OCR exhibits significant performance advantages and consistency in multi-scale character recognition tasks. MSCL-OCR achieves a highest MSR value of 0.9633, representing an 11.1% relative improvement compared to the traditional CRNN method (0.8671) and a 0.9% improvement compared to the latest TrOCR method (0.9546), validating the effectiveness of the MSAF-Net multi-scale attention feature extraction network. Specifically, MSCL-OCR achieves recognition accuracies of 99.48% and 99.67% at the Medium-large and Large-giant scales, respectively, both being the best performance among all methods. In particular, it achieves a significant improvement of 15.46 percentage points compared to CRNN at the Medium-large scale. The improvement fully demonstrates the synergistic effect of the multi-core parallel convolutional architecture and scale-aware attention fusion. Although TrOCR slightly outperforms our method at the Micro-tiny (98.53%) and Small-medium (96.25%) scales, MSCL-OCR still maintains competitive performance of 97.46% and 95.50% at these two scales. More importantly, it achieves balanced and stable performance across scales, avoiding the problem of large performance fluctuations of other methods at different scales. Of particular note is that the performance of the CRNN method drops sharply to 84.02% at the Medium-large scale. This phenomenon directly reveals the fundamental challenge faced by traditional architectures when processing large-sized characters, and further highlights the technical value and practical significance of our proposed adaptive multi-scale feature extraction strategy.

[0160] 2.2 Analysis of Scale-Aware Attention Fusion Mechanism

[0161] To gain a deeper understanding of the working mechanism of MSAF-Net, we analyzed the weight allocation pattern of scale-aware attention under different inputs, such as... Figure 4 As shown, the attention weight distribution is illustrated in different scale scenes.

[0162] Experimental results confirm the adaptability of the scale-aware attention mechanism: In the Micro-tiny scene, the system automatically allocates 81.3% ± 6.2% of the attention weights to high-resolution features (P3 layer) to capture fine character details; while in the Large-giant scene, 77.2% ± 9.6% of the weights are transferred to low-resolution features (P5 layer), focusing more on the overall structural information of the characters. The Small-medium and Medium-large scenes exhibit a progressive weight transfer feature, dominated by medium-resolution (41.1% ± 5.8%) and low-resolution (42.3% ± 12.7%) features, respectively, ensuring a smooth transition across scales. In the Mixed scene, the attention weights show a relatively balanced distribution (35.5%, 34.1%, 30.3%), reflecting the dynamic adaptability to complex multi-scale inputs. This intelligent weight allocation strategy verifies the effectiveness of the scale-aware attention fusion mechanism in handling the multi-scale challenges of industrial instrumentation.

[0163] 3. Verification Experiment on the Complexity of Instrument Spatial Layout

[0164] We conducted comparative experiments under four typical industrial display layout modes to verify the ability of content-adaptive 2D conditional position coding to handle complex spatial structures. Table 2 shows the comparison of recognition performance between fixed position coding, learned position coding and the proposed 2D conditional position coding.

[0165] As shown in Table 2, the experimental results demonstrate that 2D conditional positional coding achieves optimal performance across all layout modes: in relatively regular LED digital tube scenarios, it achieves a recognition accuracy of 96.41% ± 0.09%, representing improvements of 1.39 and 0.98 percentage points compared to fixed coding and learned coding, respectively; in LCD scenarios with significant pixelation features, it reaches 95.88% ± 0.06%, showing performance gains of 1.50 and 1.70 percentage points. Notably, the advantages of 2D conditional coding become more pronounced with increasing spatial layout complexity: in multi-row, multi-column industrial control touchscreen scenarios, the performance improvement expands to 2.00 and 2.07 percentage points, and even in the most complex scenario of heterogeneous technology mixed displays, it maintains a stable improvement of 1.74 and 1.97 percentage points. These experimental results verify that the content-adaptive mechanism can dynamically adjust the positional coding based on image features, effectively overcoming the limitations of traditional fixed coding in handling uneven character spacing, and demonstrating robust recognition performance in complex spatial layouts for industrial instrument OCR.

[0166] Table 2: Performance Verification of Complex Spatial Layout Pattern Recognition

[0167]

[0168] 4. Challenge Verification Experiment of Multi-Task Collaborative Learning for Variable-Length Sequences

[0169] To fully verify the processing capability of the three-branch collaborative architecture for sequences of variable length, as shown in Table 3, we conducted a comprehensive comparative experiment grouped by sequence length, and systematically analyzed the performance of different architecture configurations on short sequences (1-4 characters), medium sequences (5-10 characters), and long sequences (11-16 characters).

[0170] Table 3. Analysis of the differences between multi-task collaborative architecture and sequence length.

[0171]

[0172] Experimental results show that, using a single Attention architecture as a baseline, performance declines significantly with increasing sequence length, decreasing from 97.26% ± 0.10% for short sequences to 80.09% ± 0.18% for long sequences, a drop of 17.17 percentage points. Introducing the CTC auxiliary branch improves performance for all sequence length groups, with the most significant improvement in the long sequence group (82.61% vs. 80.09%), and the MGC increasing from 1.0000 to 1.0315, indicating that the alignment-independent nature of CTC is valuable for long sequence recognition. The addition of the segmentation enhancement branch further improves model performance, particularly achieving an MGC of 1.0342 in the long sequence group, validating the enhancement effect of pixel-level foreground / background discrimination on complex sequence structures. The final three-branch collaborative architecture (Attention+CTC+Seg) achieved optimal performance across all sequence length groups: 98.59% ± 0.06% (MGC = 1.0137) for short sequences, 97.70% ± 0.04% (MGC = 1.0322) for medium sequences, and 86.66% ± 0.10% (MGC = 1.0820) for long sequences. Notably, the MGC increased with sequence length, expanding from a relative gain of 1.37% for short sequences to 8.20% for long sequences. This indicates that the multi-task collaborative mechanism has a more significant differentiated supervision effect on the recognition of complex long sequences, validating the effectiveness of our multi-angle supervision strategy designed for sequences of variable length.

[0173] 5. Comparative Analysis of SOTA Methods

[0174] To objectively evaluate the overall advantages of MSCL-OCR in industrial instrument recognition tasks, we conducted a comprehensive comparison with the most advanced OCR methods currently available; Table 4 shows the overall performance of representative methods in terms of core performance indicators.

[0175] Table 4: Overall Performance Comparison of SOTA Methods in Industrial Instrument Identification Tasks

[0176] method AC (%) ↑ CER (%) ↓ MSR (%) ↑ CRNN 87.28±0.07 3.05±0.02 0.8671 RARE 94.28±0.03 0.98±0.03 0.9470 ASTER 94.35±0.02 0.99±0.02 0.9482 TrOCR 96.13±0.07 0.56±0.01 0.9554 PARSeq 97.88±0.02 0.32±0.01 0.9612 MSCL-OCR 98.50±0.02 0.28±0.01 0.9627

[0177] Experimental results demonstrate that MSCL-OCR achieves a comprehensive performance breakthrough in industrial instrument recognition tasks, reaching the best performance across all core metrics. Compared to the state-of-the-art PARSeq method, MSCL-OCR improves sequence accuracy by 0.62 percentage points (98.50% vs. 97.88%), reduces character error rate by 12.5% ​​(0.28% vs. 0.32%), and improves multi-scale robustness by 0.15 percentage points (0.9627 vs. 0.9612), showcasing comprehensive technical advantages. While MSCL-OCR and PARSeq exhibit comparable performance stability (both with a variance of ±0.02%), MSCL-OCR achieves the best results across all evaluation metrics.

[0178] Compared to the baseline CRNN method, MSCL-OCR improves sequence accuracy by 11.22 percentage points, reduces character error rate by 90.8%, and significantly increases the multi-scale robustness metric MSR from 0.8671 to 0.9627, a relative improvement of 11.0%, validating the effectiveness of the multi-scale spatial awareness mechanism. Even compared to the advanced Transformer architecture TrOCR, MSCL-OCR still achieves an accuracy improvement of 2.37 percentage points and a 50.0% reduction in error rate, fully demonstrating the technical value of the multi-task collaborative learning strategy.

[0179] 6. Ablation test

[0180] To accurately quantify the independent contributions and synergistic effects of each technical module, we designed a systematic ablation experiment to progressively verify the specific value of MSAF-Net, Content Adaptive 2D Conditional Position Coding (C-Pe), and Three-Branch Collaborative Decoding Architecture (TCD-CSM). The experimental results are shown in Table 5, where “√” indicates that the module is enabled and “×” indicates that the module is not enabled.

[0181] Table 5: Impact Analysis of Each Technical Module on MSCL-OCR

[0182] Model Configuration MSAF-Net C-Pe TCD-CSM AC (%) ↑ CER (%) ↓ MSR↑ MGC↑ TransformerOCR × × × 97.10±0.13 0.55±0.02 0.9122 1.0 +MSAF-Net √ × × 97.62±0.05 0.50±0.01 0.9564 1.0 +C-Pe √ √ × 98.33±0.07 0.33±0.01 0.9574 1.0 MSCL-OCR √ √ √ 98.50±0.02 0.28±0.01 0.9633 1.0017

[0183] Experimental results show that each module makes a significant positive contribution to the overall performance. The baseline Transformer OCR architecture achieves a recognition accuracy of 97.10% ± 0.13% and a character error rate of 0.55% ± 0.02%. After introducing the MSAF-Net multi-scale attention feature extraction network, the recognition accuracy improves to 97.62% ± 0.05%, the character error rate decreases to 0.50% ± 0.01%, and the multi-scale robustness index (MSR) significantly improves from 0.9122 to 0.9564, verifying the effectiveness of the multi-scale feature fusion mechanism. Furthermore, adding content-adaptive 2D conditional position coding further improves the performance to 9. An accuracy of 8.33% ± 0.07% and an error rate of 0.33% ± 0.01% were achieved, with a slight improvement in MSR to 0.9574, demonstrating the unique value of conditional positional encoding in handling complex spatial layouts. The complete MSCL-OCR architecture, through the integration of a three-branch collaborative decoding module, ultimately achieved a recognition accuracy of 98.50% ± 0.02%, a character error rate of 0.28% ± 0.01%, and an MSR of 0.9633. The multi-task gain coefficient (MGC) reached 1.0017, achieving a superlinear collaborative gain effect. Notably, the introduction of each module was accompanied by a significant reduction in performance variance, gradually decreasing from ± 0.13% at the baseline to ± 0.02% at the final level, indicating that each technical component not only improved the average performance but also enhanced the stability and reliability of the model.

[0184] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims

1. A method for instrument optical character recognition with multi-scale spatial perception and task-cooperative learning, characterized in that, The method comprises the following steps: S1: data augmentation and preprocessing of uniform batch size for the pictures in the data set; S2: the preprocessed pictures are sent to an image feature extractor for multi-scale visual feature extraction; S3: character position information is extracted according to the multi-scale visual features to embed position information in the multi-scale visual features; S4: inputting the multi-scale visual features containing the position information into a Transformer encoder to learn global features to obtain shared encoding features F SEF ; S5: F by segmentation seg, text alignment CTC and decoder Attention SEF loss supervision, optimizing shared encoding feature representation; S6: the model is optimized end-to-end until the model is fitted and the optimal result is achieved by combining the multi-scale feature learning loss, the pixel segmentation loss, the alignment-independent loss and the decoding prediction loss.

2. The multi-scale space-aware and task-cooperative learning instrument optical character recognition method according to claim 1, wherein, The data augmentation and preprocessing of uniform batch size comprise the following steps: A 1x1 convolutional layer is used to project the number of channels of each original feature map {P i} uniformly to the hidden dimension d model of the model. The spatial resolution of the original feature map {P i} is adjusted to the target size by bilinear interpolation to obtain the preprocessed target feature map The specific calculation process is as follows: Let the size of the original feature map {P i} be (H i , W i ), and the target size be (H t arget, W t arget), then the scaling ratio is: scale h =H t arget / H i , scale w =W t arget / W i , for any position (x', y') in the target feature map, its corresponding original coordinates are: x=x' / scale w , y=y' / scale h , the bilinear interpolation is weighted calculated by four nearest neighbor pixel values, first, the coordinates of the four nearest neighbor pixels are determined, the calculation formula is: wherein x1 represents a value obtained by rounding down x, x2 represents a value obtained by adding 1 to x1, y1 represents a value obtained by rounding down y, and y2 represents a value obtained by adding 1 to y1, represents a rounding down operation; The pixel value after interpolation is calculated according to the following formula: Among them, P i (x1, y1), P i (x2, u1), P i (x1, y2) and P i (x2, y2) represents the pixel values ​​of the four nearest neighbors in the original feature map, P i ′(x′, y′) represents the interpolated pixel value at position (x′, y′) in the target feature map; this interpolation process is used to combine all the original feature maps {P i Adjust to the same spatial resolution (H t arget, W t target feature map 3. The multi-scale space-aware and task-cooperative learning instrument optical character recognition method according to claim 1, wherein, The multi-scale visual feature extraction comprises the following steps: for each aligned target feature map extract its global context descriptor where i is an integer equal to 1 to N; all scales of context descriptors are dimensionally concatenated by Concat function, input into scale attention multi-layer perception, learning to predict normalized attention weight distribution a = a1, a2, …, a N The calculation formula is: a = Softmax(SA_MLP(Concat(s1, s2,..., s N ))) ; wherein, Softmax() is a probability normalization function, SA_MLP() is a scale attention multi-layer perceptron, Concat() represents a dimension concatenation function, and N represents the number of fused scales; Finally, the fused target feature map F fused is generated by dynamically weighting and summing each scale feature map, and the calculation formula is: wherein a i denotes a weighting coefficient of the i-th scale feature map, denotes a feature map of the i-th scale.

4. The multi-scale space-aware and task-cooperative learning instrument optical character recognition method according to claim 1, wherein, The S3 comprises the following steps: Adopting content-adaptive 2D conditional positional encoding CPE, which is to output two-dimensional feature map F by feature extractor enc As the input of convolutional network, dynamically generate 2D position encoding map PE closely related to the current image content features cond ; then element-wise addition is performed with F enc To form the initial state F of the sequence representation finally sent into the Transformer encoder enc′ The calculation formula is: F enc′ = F enc + PE cond ; where + is an element-wise addition symbol, F enc is the fused target feature map F fused , PE cond denotes a 2D position encoding map.

5. The multi-scale space-aware and task-cooperative learning instrument optical character recognition method according to claim 1, wherein, The S4 comprises the following steps: Transformer encoder transforms the initial state F represented by a sequence enc′ into shared encoded features where H, W and D are height, width and feature dimension respectively, as a unified feature basis for all downstream tasks; TCLM module constructs a quadruple collaborative supervision framework, and scale-aware loss guides optimal multi-scale fusion in the feature extraction stage; segmentation, attention and CTC three parallel branches respectively provide F SEF with differentiated supervision signals.

6. The multi-scale space-aware and task-cooperative learning instrument optical character recognition method according to claim 1, wherein, The S5 comprises the following steps: S5-1: dynamically assigning loss weights according to sample scale categories to ensure that different scale characters are given differential attention; S5-2: to enhance the adaptive ability of the model to multi-scale features, a scale-aware attention supervision mechanism is introduced; S5-3: three parallel branches cooperatively optimize shared encoding features through different supervision signals, forming a complementary learning mechanism.

7. The multi-scale space-aware and task-cooperative learning instrument optical character recognition method according to claim 6, wherein, The differential attention to different scale characters comprises the following steps: Dynamic weight of sample i in the tth training round The calculation formula is: where a = 0.8 and β = 1.5 are scaling factors, W base = 1.0 is a base weight, τ high = 0.95 and τ low = 0.85 are performance thresholds, represents the class performance of sample i, cat i represents the scale class of sample i, represents the class performance statistics of the previous round.

8. The multi-scale space-aware and task-cooperative learning instrument optical character recognition method according to claim 6, wherein, The scale-aware attention supervision mechanism comprises the following steps: its supervision loss The calculation formula is: where λ ent and λ div are weight coefficients balancing different loss terms, λ ent = 0.1, λ div = 0.05; is the supervision loss, is the divergence loss term, is the entropy regularization term, is the diversity loss term; The supervision loss is composed of three complementary items, including: diversity loss term: main supervisory signal by enforcing predictive attention weights alignment with the target distribution a target is achieved by aligning with the target distribution a where KL (||) denotes the Kullback-Leibler divergence alignment, a pred denotes the predicted attention weight vector, a target denotes the target attention weight vector, a target,i and αpred,i denote the i-th element in the a target and a pred vector, respectively. Entropy Regularization: To prevent the generation of overconfident attention distributions, we employ entropy regularization, which is computed as: Diversity loss term: To encourage batch-wise attention diversity, we minimize the deviation from a uniform distribution, which is computed as: where, represents batch attention mean, u = [1 / 3, 1 / 3, 1 / 3] T is a uniform distribution vector, represents the predicted attention weight vector for the b-th sample, B represents the batch size, represents the square calculation of the L2 norm.

9. The multi-scale space-aware and task-cooperative learning instrument optical character recognition method according to claim 6, characterized in that, The three parallel branches comprise the following steps: wherein, the branch calculation formula for prompting the encoder to learn a time-robust feature representation is as follows: wherein, denotes prompting the encoder to learn a timing-robust feature representation, denotes a path of all possible alignments of the target sequence Y, T denotes a time step, P ctc (π t |F SEF ) denotes a CTC probability at time t, π t denotes the value of the path π at time t; The segmentation branch enhances the spatial discrimination ability through pixel-level segmentation, especially for accurate character boundary positioning, and the calculation formula is as follows: wherein, represents that the split branch enhances the spatial discrimination ability through pixel-level segmentation, is a focal loss for handling class imbalance, is a U-shaped spatial regularization penalty term; λ u = 1.0 represents the penalty term weight; To prevent the model from producing trivial solutions of full foreground or full background, the attention branch serves as the main recognition path and provides semantic-level supervision through autoregressive decoding, and the calculation formula is as follows: wherein, denotes the attention branch, L is the target sequence length, is the true label at position t, denotes all preceding labels, log P(|) denotes the probability logarithm, F SEF denotes the shared encoding features.

10. The multi-scale space-aware and task-cooperative learning instrument optical character recognition method according to claim 1, wherein, The S6 comprises the following steps: The multi-task cooperative mechanism realizes end-to-end optimization of total value through a weighted joint loss function The calculation formula is: where λ att = 1.0 is the attention weight, denotes the attention branch; λ ctc = 0.2 is the robust feature weight, denotes the learning of a temporally robust feature representation by the encoder; λ seg = 0.2 is the discriminative ability weight, denotes the segmentation branch; λ scale = 0.2 is the supervision loss weight, denotes the supervision loss.

Images