Robustness end-to-end method for optical character recognition of industrial instruments and meters under complex working conditions

Abstract

The invention provides a robust end-to-end method for optical character recognition of industrial instruments and meters under complex working conditions. The method comprises the following steps: S1, preprocessing pictures in a data set; s2, inputting the preprocessed picture into a channel normalization selection module for processing; s3, carrying out inclination angle detection on the processed image, and judging whether the inclination angle is greater than a preset threshold value theta; if the angle is smaller than or equal to the threshold value, entering the next step; if yes, geometric transformation correction is carried out on the image; s4, the corrected image is sent to a lightweight target detection model; s5, expanding the frame of each text area detected in the step S4 according to a proportion; s6, inputting the new image into an optical character recognition system; s7, processing abnormal conditions in the process; s8, outputting the structured text information; the problem of accurate detection of multi-scale and low-quality text targets in an instrument image and a character target complex image in an industrial scene is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of optical character recognition of industrial instruments, and particularly relates to a robust end-to-end method for optical character recognition of industrial instruments under complex working conditions. BACKGROUND

[0002] Optical instrument character recognition is a key technology for industrial automation and intelligent transformation, and its core is to replace traditional manual inspection with a deep learning algorithm to realize automatic collection and structured analysis of instrument information. However, in actual operation, industrial site instrument inspection is usually completed by an operator controlling a portable image device, which is easy to cause non-uniformity in the collected image in terms of angle deviation, light fluctuation, and object distance change. This non-uniformity leads to the following four challenges in the OCR (optical character recognition) of industrial scene optical instrument labels: the instrument image has multiple random disturbances, such as severe illumination variation (e.g., highlight overflow or dark part loss), significant angle deviation, and long-distance imaging, which will degrade the resolution and chroma fidelity of the image quality; in addition, the target object has heterogeneous characteristics (e.g., material reflection difference), and the label of the industrial instrument is difficult to read; the label of an instrument that has been in operation for a long time becomes difficult to identify due to long-term environmental erosion; and the huge image quality difference makes it difficult for a deep model to capture the internal features of the instrument label with generalization ability.

[0003] Industrial optical instrument images are inclined, and when a handheld device is used for shooting, the instrument image is easy to have angle deviation, which leads to the inclination of the text region. This inclination changes the geometry and arrangement of the characters, making character segmentation and detection extremely difficult. In extreme cases, the characters can be severely distorted, making it difficult for the model to accurately identify them.

[0004] The character detection boxes of industrial instruments are redundant and overlapped. Long-term service instruments are affected by mechanical vibration, oil pollution, and metal oxidation, which easily causes character displacement, stroke connection, and reflection artifacts, leading to spatial coupling interference of the deep learning model due to local feature confusion, resulting in distortion of the character topology. SUMMARY

[0005] The purpose of the present application is to solve the problem of image quality degradation caused by severe illumination variation, highlight overflow, dark part loss, and unknown light conditions in industrial scene instrument images, and in particular to introduce a robust end-to-end method for optical character recognition of industrial instruments under complex working conditions.

[0006] In order to achieve the above-mentioned purpose of the present application, a robust end-to-end method for optical character recognition of industrial instruments under complex working conditions is provided, which comprises the following steps:

[0007] S1: Preprocessing the pictures in the data set;

[0008] S2: input the preprocessed picture into a channel normalization selection module for processing;

[0009] S3: detect the tilt angle of the processed image, and determine whether the tilt angle is greater than a preset threshold; if the angle is less than or equal to the threshold, the image will directly enter the next step; if the angle is greater than the threshold, the image will be corrected by geometric transformation and then enter the next step;

[0010] S4: send the image obtained in S3 to a lightweight target detection model for detection;

[0011] S5: expand and reorganize the bounding box of each text region detected in S4 according to a proportion;

[0012] S6: input the new image after expansion and reorganization into an optical character recognition system;

[0013] S7: handle abnormal situations that occur during the recognition process;

[0014] S8: output structured text information, completing the end-to-end recognition process from the original image to data parsing.

[0015] Preferably, the channel normalization selection module comprises:

[0016] a dynamic adaptive instance light normalization module and a dynamic adaptive gating module.

[0017] Further, the dynamic adaptive instance light normalization module comprises:

[0018] The dynamic adaptive instance light normalization module has the following process:

[0019]

[0020] wherein represents a channel-wise multiplication, x represents a three-dimensional feature map, x' represents the feature of the three-dimensional feature map x after DAIN normalization processing, DAIN() represents DAIN normalization processing, γ, represents a non-fixed learnable parameter dynamically generated from the input feature, represents a real set on the C channel; δ is a very small positive real number, max() represents taking the maximum value, σ c (x) represents the channel standard deviation of the three-dimensional feature map x; μ c (x) represents the channel mean of the three-dimensional feature map x, and the calculation method is as follows:

[0021]

[0022] wherein H represents the height of the three-dimensional feature map x, W represents the width of the three-dimensional feature map x, and xi,j,c represents a specific value in the three-dimensional feature map x;

[0023] σ c (x) represents the channel standard deviation of the three-dimensional feature map x, which is calculated as:

[0024]

[0025] where each channel c∈{1,2,...,C} is calculated independently, ensuring that the inter-instance illumination difference is suppressed, and C represents the number of channels of the three-dimensional feature map x;

[0026] The calculation of γ and β is as follows:

[0027] γ=W γ ·GAP(x),β=W β ·GAP(x)

[0028]

[0029] where W γ , is a learnable weight matrix, which reduces redundancy through sparse initialization; is a set of C rows and C columns of real numbers, and GAP() represents global average pooling; DAIN normalization can reduce the illumination difference between instances, so the features x′ normalized by DAIN have robust characteristics that are not related to the lighting conditions.

[0030] Further, the dynamic adaptive gating module comprises:

[0031] The dynamic adaptive gating module processing flow is as follows:

[0032] x n+1 =γ(g)⊙x n ′+(1-γ(g))⊙x n

[0033] where γ(g) is the channel attention weight, and ⊙ represents the channel-wise multiplication, x n represents the original feature map input to the channel normalization selection module, x n ′ represents the feature map after normalization processing of x n , and x n+1 represents the generated target feature map.

[0034] The calculation of the channel attention weight γ(g) is as follows:

[0035] γ(g)=σ(W'·GAP(g))

[0036] Where σ() is the Sigmoid function, g represents a binary indicator along the channel dimension, W' represents a weight matrix, and GAP() represents global average pooling.

[0037] Therefore, the generated target feature map x n+1 By eliminating the effects of illumination, a constant representation is obtained, along with unchanged channels, while preserving crucial information to support accurate reconstruction. Specifically, the gating operation should be differentiable and able to limit the output to 0 or 1 for channel selection. The gating module is designed as shown below, represented as follows:

[0038] g=G(α x )=σ(ω·α x )

[0039] Where ω is the scaling factor, α x It is an intermediate vector generated from the input features, where σ() is the Sigmoid function and G() represents a gating function. Specifically, firstly, in order to construct the intermediate vector α associated with the input features... x The spatial dimension of the features is compressed to a single pixel scale through global average pooling, and then the final representation is obtained through a linear transformation. Finally, a scaling factor ω is introduced into the gating function.

[0040] When α x >>0, resulting in g = G(α) x )≈1; when α x << 0 yields g = G(α) x Therefore, the designed function G(α) ≈ 0; x Able to convert any α x Effectively mapped to the interval [0,1].

[0041] Furthermore, the image tilt angle detection includes:

[0042] Step 1: Input the original image I and obtain the low threshold T low and high threshold T high Used to determine whether a pixel belongs to an edge; obtain the Hough parameter ρ res =1,θ res =1°, accumulator threshold T acc Minimum line segment length L min and maximum gap G max ;

[0043] Image I is obtained by grayscale conversion of the original image I. gray ;

[0044] Step 2: Initialize the Hough accumulator and extract image I. gray Each edge point (x i,y i );

[0045] For θ∈[0°,180°) with ρ res and θ res Traverse for step size and angle;

[0046] The formula for calculating ρ is as follows:

[0047] ρ=x i cosθ+y i sinθ

[0048] Where, x i and y i The x and y coordinates of the edge point are represented, where θ represents the angle of the edge point and ρ represents the radial distance in the polar coordinate system.

[0049] Secondly, the input to the Hough accumulator yields a candidate straight-line threshold A(ρ). j ,θ j );

[0050] Step 3: Extract candidate lines, if A(ρ j ,θ j )≥T acc Record (ρ) j ,θ j );

[0051] Where, A(ρ) j ,θ j ) represents the candidate line threshold, T acc ρ represents the accumulator threshold. j and θ j These represent the step size of the Hough accumulator in the distance and angle dimensions, respectively;

[0052] Step 4: Determine the length and gap of each candidate line. If the line segment length is ≥ L... min And the gap ≤ G max If so, then retain it;

[0053] Step 5: Construct an angle set Θ = {θ} j};Θ filtered ={θ∣-45°<θ<45° or 135°<θ<180°}; θ skew =Median(Θ) filtered );

[0054] Where Θ represents an angle set used to store the angles of all detected candidate lines; Θ filtered This represents the filtered set of angles; Median() means taking the median, θ skew Indicates the tilt angle;

[0055] Step 6: Calculate the tilt angle θ skew And output it.

[0056] Furthermore, the lightweight target detection model includes:

[0057] The module includes a character area detection module for optical instruments and meters, a structured text module, and a character detection module for optical instruments and meters.

[0058] Furthermore, the optical instrument character area detection module includes:

[0059] The localization loss function is optimized to address challenges in industrial scenarios, such as image tilt and target scale differences caused by varying viewing angles and long-distance shooting, as well as character blurring, glare, and stroke adhesion caused by uneven lighting and equipment aging. Considering that the CIoU loss function can take into account the aspect ratio of the detection box, it is used as the core optimization criterion for image annotation target detection tasks, as shown below:

[0060]

[0061] in, CIoU represents the loss function, and IoU represents the intersection-over-union ratio. This represents the penalty term in the CIoU loss function, used to measure the positional and shape differences between the predicted and ground truth boxes. Its calculation method is as follows:

[0062]

[0063] Where τ is the adaptive weight coefficient, ρ 2 (b,b gt ) represents the squared Euclidean distance between the predicted bounding box and the ground truth bounding box, used to measure the difference in their center point positions; d 2 represents the square of the diagonal length of the smallest closure covering both the predicted and ground truth boxes, used to normalize the Euclidean distance. v measures the similarity in aspect ratio between the predicted and ground truth boxes, and is calculated as follows:

[0064]

[0065] Where w and h represent the width and height of the prediction box, w gt and h gt These represent the width and height of the true bounding box, arctan represents the arctangent function, and π represents pi; τ is calculated as follows:

[0066]

[0067] The partial derivatives of v with respect to the two variables w and h are calculated as follows:

[0068]

[0069] in, Let r represent the partial derivative, r be the calculation correction term, and ∈ be a very small smoothing constant used to prevent calculation errors caused by a zero denominator;

[0070] Finally, a targeted attribute-weighted loss was designed, and a suitable loss function, Attribute-Weighted, was developed. loss :

[0071]

[0072] Among them, f α f represents the class balancing factor, responsible for balancing the class distribution. ο f represents the target scaling factor; δ This represents the attribute balance factor, which is specifically designed to handle complex image attributes in industrial scenarios. This indicates reducing the loss weight factor for easily classified samples; Dice loss The loss function used in image segmentation and classification tasks is calculated as follows:

[0073]

[0074] Among them, y n Indicates the real label, p n This refers to the model output calculated using the sigmoid function;

[0075] f α =α·y n +(1-α)·(1-y n )

[0076] f ο =ο·y n +(1-ο)·(1-y n )

[0077]

[0078]

[0079] Where α and For hyperparameters, ο represents the bounding box size factor; ο controls the target scale factor f. ο The weighting suppression strength for large targets is determined by a larger value, which will significantly reduce the loss weight of large targets, thus relatively amplifying the focus on small targets. Finally, a new factor is designed. Classify and label samples as one or more attribute classes; p represents the proportion of samples of class k;t The confidence score of the model for the correct category is calculated as follows:

[0080] p t =y n ·p n +(1-y n )·(1-p n )

[0081] The calculation method for ο is as follows:

[0082]

[0083] Among them, h min and w min This indicates the minimum height and width to be set.

[0084] Furthermore, the structured text module includes:

[0085] Because the acquired image may be tilted, a text frame based on a specific format is introduced. The borders of the detected text regions are expanded according to the following formula:

[0086] x′ min =x min -x l ×(Δy)×d l

[0087] y′ min =y min -y t ×(Δy)×d t

[0088] x′ max =x max +x r ×(Δy)×d r

[0089] y′ max =y max +y d ×(Δy)×d d

[0090] Δy=y max -y min

[0091] Among them, (x min ,y min ),(x max ,y max () represents the coordinates of the top left and bottom right corners of the border; x l and x r y represents the left and right expansion scaling factor. t and y dd represents the scaling factor for vertical expansion. l and d r d represents the left and right expansion direction factor. t and d d This represents the vertical expansion direction factor; the values ​​of these parameters are adjusted according to the stage two target detection model; Δy is the proportion of the expanded object, while (x′) represents the vertical expansion direction factor. min ,y′ min ),(x′ max ,y′ max The ) is used to specify the location of the extended object.

[0092] Furthermore, the optical instrument character detection module includes:

[0093] The optical instrument character detection module improves its ability to detect text at different scales by introducing an attention mechanism and multi-scale feature fusion technology; the specific calculation process is as follows:

[0094] P=σ(F(I))

[0095]

[0096] F(I)=Conv(SwinTransformer(I))

[0097] Where P is the generated probability map, I represents the input image, F(I) represents the final feature map, σ() represents the sigmoid function, M represents the binary map, exp represents the exponential operation with the natural constant e as the base, -λ represents a scaling factor that controls the smoothness of the function, and T represents the learnable threshold map. Represents the loss function. To predict the loss between the probability map P and the true label, μ and ξ are hyperparameters. It is the loss that predicts the relationship between the binary image M and the true label. It is a regularization term for the threshold map T. Conv() indicates convolutional layer processing, and SwinTransformer() indicates SwinTransformer network processing.

[0098] Furthermore, S7 includes:

[0099] There is a small probability of overlapping bounding boxes occurring during object detection; therefore, an overlapping bounding box correction module was designed to solve this problem, including:

[0100] Step 1: Input the original image I and detect the target boxes in the image to obtain the target box set B;

[0101] Step 2: Initialize the program; if |B|≥2, then for the target box B... i Bj ∈B performs the following operation;

[0102] Among them, B i and B j For elements in set B;

[0103] Calculate B i and B j The overlap is calculated using the following formula:

[0104]

[0105] Among them, IoU ij B i and B j The intersection-union ratio, Area(B) i ∩B j ) represents element B i and B j The intersection of Area(B) i ∪B j ) represents element B i and B j The union of the parts;

[0106] If IoU ij >T overlap Then calculate the center distance d ij The calculation formula is as follows:

[0107] d ij =‖Center(B i )-Center(B j )‖2

[0108] Where, d ij Center(B) represents the distance between centers. i ) and Center (B j ) represents element B i and B j The center coordinates, |||2 is used for center distance calculation, T overlap The threshold value for the overlapping portion of the target bounding box;

[0109] If d ij <T distance Then merge element B i and B j A new target box B is obtained. new The calculation formula is as follows:

[0110] B new =Merge(B i B j )

[0111] Among them, B new Merge() indicates merging newly generated target boxes, and T represents the merge operation. distance The threshold value for the set center distance;

[0112] And update set B, using the following formula:

[0113] B = {B i B j}∪{B new}

[0114] Step 3: Detect and merge set B to obtain a new target set B merged ;

[0115] Step 4: Output a new set of bounding boxes B merged .

[0116] In summary, by utilizing the above technical solutions, the robust end-to-end method for optical character recognition of industrial instruments under complex working conditions proposed in this invention can effectively remove illumination interference factors from the input data through the channel normalization selection module, while adaptively retaining discriminative features related to illumination conditions. It can improve the generalization ability of the entire framework for images obtained by instruments in industrial scenarios under unknown illumination conditions, enabling it to exhibit good robustness in a wide range of unknown illumination scenarios under limited illumination conditions. By adaptively adjusting image scaling and intelligently and dynamically learning bounding box anchor points, it improves the detection capability for different sizes and dimensions. Attached Figure Description

[0117] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0118] Figure 1 This is a diagram of the end-to-end optical character detection and recognition framework of the present invention.

[0119] Figure 2 This is a flowchart of the invention.

[0120] Figure 3 This is a schematic diagram of the channel normalization selection module of the present invention.

[0121] Figure 4 This is a schematic diagram of the text direction determination method of the present invention.

[0122] Figure 5 This is a schematic diagram of the process of expanding the target area of ​​the present invention.

[0123] Figure 6 This is a comparison chart of the frame rate performance of the text detection algorithm for industrial datasets in this invention.

[0124] Figure 7 This is a schematic diagram illustrating the effect of the channel normalization selection module of the present invention.

[0125] Figure 8 This is the deviation analysis diagram for angle calculation in this invention.

[0126] Figure 9 This invention relates to the analysis of α and γ hyperparameters.

[0127] Figure 10 These are comparison images of detection examples of the present invention. Detailed Implementation

[0128] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0129] like Figure 1 The diagram shown illustrates a robust end-to-end method for optical character recognition in industrial instruments under complex operating conditions. The framework for end-to-end optical character detection and recognition includes:

[0130] The preprocessed image is input into the framework and processed through the channel normalization selection module. Dynamic adaptive instance normalization in the channel normalization module is used to normalize the image feature channels, enhancing adaptability to illumination changes. Next, dynamic adaptive gating in the channel normalization selection module enables dynamic adaptive selection of channels, generating new feature maps. By eliminating the influence of illumination, a constant representation is obtained, resulting in unchanged channels and preserving key information to support accurate reconstruction. For single or multiple targets, tilt angle detection is performed to determine if the tilt angle is greater than a preset threshold θ (θ = 25°). If the angle is less than or equal to the threshold, the text box border is proportionally expanded. If it is greater than the threshold, the image is geometrically transformed and corrected using a tilt angle calculation method based on Hough Transform before proportional expansion. The expanded and reconstructed image is input into an independent optical character recognition (OCR) system for recognizing optical characters and other key text. To address the issue of generating multiple highly overlapping prediction boxes for the same target, an overlapping box correction module is designed.

[0131] like Figure 2 The diagram shows a robust end-to-end method for optical character recognition in industrial instruments under complex operating conditions, including:

[0132] S1: Preprocess the images in the dataset;

[0133] Specifically, including:

[0134] Data cleaning: Inspect and remove images in the dataset that are of very low quality (e.g., excessively blurry, severely occluded, or completely unrecognizable);

[0135] Image size normalization: Adjust all input images to a uniform size of 640x640 pixels to meet the input dimension requirements of subsequent deep learning models;

[0136] Data Augmentation: Geometric Transformation: Perform random rotation, scaling, translation, and cropping operations on the image to simulate different shooting angles and distances, increasing the model's robustness to changes in position and size;

[0137] Color / Lighting Transformation: Randomly adjust the brightness, contrast, and saturation of the image, or add Gaussian noise to simulate complex and ever-changing lighting conditions on site;

[0138] S2: Input the preprocessed image into the frame, encode it, process it through the channel normalization selection (CNSel) module, and then decode and output it;

[0139] Specifically, the channel normalization selection module, such as Figure 3 As shown, it includes:

[0140] Dynamic adaptive instance light normalization module and dynamic adaptive gating module;

[0141] The feature channels undergo normalization preprocessing to eliminate distribution bias and adapt to the computational requirements of subsequent differential channel selection. Dynamic adaptive instance normalization (DAIN) is used to enhance adaptability to illumination changes. Given features of shape H×W×C, the DAIN process is as follows:

[0142]

[0143] Where ⊙ denotes channel-wise multiplication, x represents a 3D feature map, x′ represents the feature map x after DAIN normalization, DAIN() represents DAIN normalization, and γ, This represents non-fixed learnable parameters that are dynamically generated from input features. Let δ represent a set of real numbers in dimension C; δ is a very small positive real number, max() represents taking the maximum value, and σ c (x) represents the channel standard deviation of the 3D feature map x; μ c (x) represents the channel mean of the 3D feature map x, which is calculated as follows:

[0144]

[0145] Where H represents the height of the 3D feature map x, W represents the width of the 3D feature map x, and x i,j,c This represents a specific numerical value in the three-dimensional feature map x;

[0146] σ c (x) represents the channel standard deviation of the 3D feature map x, which is calculated as follows:

[0147]

[0148] Each channel c∈{1,2,...,C} is computed independently to ensure that the lighting differences between instances are suppressed, and C represents the number of channels in the 3D feature map x.

[0149] γ and β represent non-fixed learnable parameters dynamically generated from input features, and their calculation method is as follows:

[0150] γ=W γ ·GAP(x),β=W β ·GAP(x)

[0151]

[0152] Among them, W γ , It is a learnable weight matrix, which reduces redundancy through sparse initialization; GAP() represents global average pooling, where C is a set of real numbers and C is a set of real numbers. DAIN normalization can reduce the lighting differences between instances, so the feature x′ normalized by DAIN has robust properties that are independent of lighting conditions.

[0153] To achieve dynamic adaptive channel selection with minimal changes to the network structure, a dynamic adaptive gating module is introduced for channel selection. This module outputs a series of binary indicators to combine the normalized and original channels along the channel dimension, which can be represented as:

[0154] x n+1 =γ(g)⊙x n ′+(1-γ(g))⊙x n

[0155] Where γ(g) is the channel attention weight, ⊙ represents channel-wise multiplication, and x n This represents the original feature map input to the channel normalization selection module, x. n ′ represents x n The feature map after normalization, x n+1 This represents the generated target feature map;

[0156] The channel attention weight γ(g) is calculated as follows:

[0157] γ(g)=σ(W'·GAP(g))

[0158] Where σ() is the Sigmoid function, g represents a binary indicator along the channel dimension, W' represents a weight matrix, and GAP() represents global average pooling.

[0159] Therefore, the generated target feature map x n+1 By eliminating the effects of illumination, a constant representation is obtained, along with unchanged channels, while preserving crucial information to support accurate reconstruction. Specifically, the gating operation should be differentiable and able to limit the output to 0 or 1 for channel selection. The gating module is designed as shown below, represented as follows:

[0160] g=G(α x )=σ(ω·α x )

[0161] Where ω is the scaling factor, α x It is an intermediate vector generated from the input features, where σ() is the Sigmoid function and G() represents a gating function. Specifically, firstly, in order to construct the intermediate vector α associated with the input features... x Global average pooling is used to compress the spatial dimension of features to a single pixel scale, followed by a linear transformation to obtain the final representation. Then, a scaling factor ω is introduced into the gating function, such as... Figure 2 As shown;

[0162] When α x >>0, resulting in g = G(α) x )≈1; when α x << 0, thus obtaining g = G(α) x Therefore, the designed function G(α) ≈ 0; x Able to convert any α x It effectively maps to the [0,1] interval, thus realizing a gating mechanism that does not require manual threshold setting; at the same time, compared with traditional methods, it can avoid the problem of excessive retention in the channel selection process while maintaining gradient stability, so that the network can achieve a better balance between feature selection ability and generalization ability.

[0163] S3: Perform tilt angle detection on the processed image to determine if its tilt angle is greater than a preset threshold θ (θ = 25°). If the angle is less than or equal to the threshold, the image will proceed directly to the next step; if it is greater than the threshold, the tilt angle calculation method based on Hough transform will be called to perform geometric transformation correction on the image before proceeding to the next step.

[0164] Specifically, this includes: for the remaining few images with large tilt angles, using an optional perspective transformation method for text detection in the Optical Character Recognition (OCR) system; optimizing the alignment and standardization of text regions through geometric transformations to improve the overall recognition accuracy; in the image tilt correction task, this study uses the Hough transform as the core method to calculate the tilt angle, accurately identify label images with large tilt angles, and uses perspective transformation for correction to enhance the stability and flexibility of the OCR system in complex scenes; the steps for calculating the image tilt angle based on the Hough transform are as follows:

[0165] Step 1: Input the original image I and obtain the low threshold T low and high threshold T high Used to determine whether a pixel belongs to an edge; obtain the Hough parameter ρ res =1,θ res =1°, accumulator threshold T acc Minimum line segment length L min and maximum gap G max ;

[0166] Image I is obtained by grayscale conversion of the original image I. gray ;

[0167] Step 2: Initialize the Hough accumulator and extract image I. gray Each edge point (x i ,y i );

[0168] For θ∈[0°,180°) with ρ res and θ res Traverse for step size and angle;

[0169] The formula for calculating ρ is as follows:

[0170] ρ=x i cosθ+y i sinθ

[0171] Where, x i and y i The x and y coordinates of the edge point are represented, where θ represents the angle of the edge point and ρ represents the radial distance in the polar coordinate system.

[0172] Secondly, the input to the Hough accumulator yields a candidate straight-line threshold A(ρ). j ,θ j );

[0173] Step 3: Extract candidate lines, if A(ρ j ,θ j )≥T acc Record (ρ) j,θ j );

[0174] Where, A(ρ) j ,θ j ) represents the candidate line threshold, T acc ρ represents the accumulator threshold. j and θ j These represent the step size of the Hough accumulator in the distance and angle dimensions, respectively;

[0175] Step 4: Determine the length and gap of each candidate line. If the line segment length is ≥ L... min And the gap ≤ G max If so, then retain it;

[0176] Step 5: Construct an angle set Θ = {θ} j};Θ filtered ={θ∣-45°<θ<45° or 135°<θ<180°}; θ skew =Median(Θ) filtered );

[0177] Where Θ represents an angle set used to store the angles of all detected candidate lines; Θ filtered This represents the filtered set of angles; Median() means taking the median, θ skew Indicates the tilt angle;

[0178] Step 6: Calculate the tilt angle θ skew And output it.

[0179] S4, feed the corrected image into an improved lightweight object detection model;

[0180] Specifically, lightweight object detection models include:

[0181] The optical instrument character area detection module, the structured text module, and the optical instrument character detection module are all included.

[0182] The instrument and meter character area detection module includes:

[0183] The application of optical character detection and recognition in industrial scenarios focuses on how to stably and accurately extract character regions in environments with uneven lighting and complex backgrounds, especially since characters on dashboards often vary significantly in size, color, and font representation. Adaptive image scaling and intelligent dynamic learning of bounding box anchor points improve the detection capability for different sizes (especially small targets). Most importantly, the combination of the ConsistentDualAssignment mechanism and the One-to-One DetectionHead removes redundant prediction boxes in a one-to-one manner during the inference phase. Its lightweight neck structure and efficient feature fusion capabilities offer advantages in both model performance and inference speed. Therefore, this advanced real-time detector is chosen as the foundational architecture, and a unique loss function is designed around it to significantly improve accuracy in instrument numerical detection tasks. Simultaneously, an appropriate attribute-weighted loss function is designed. loss It is used to improve the accuracy of instrument numerical detection in complex environments.

[0184] The localization loss function is optimized to address challenges in industrial scenarios, such as image tilt and target scale differences caused by varying viewing angles and long-distance shooting, as well as character blurring, glare, and stroke adhesion caused by uneven lighting and equipment aging. Considering that the CIOU loss function can take into account the aspect ratio of the detection box, it is used as the core optimization criterion for image annotation target detection tasks, as shown below:

[0185]

[0186] in, CIoU represents the loss function, and IoU represents the intersection-over-union ratio. This represents the penalty term in the CIoU loss function, used to measure the positional and shape differences between the predicted and ground truth boxes. Its calculation method is as follows:

[0187]

[0188] Where τ is the adaptive weight coefficient, ρ 2 (b,b gt ) represents the squared Euclidean distance between the predicted bounding box and the ground truth bounding box, used to measure the difference in their center point positions; d 2 represents the square of the diagonal length of the smallest closure covering both the predicted and ground truth boxes, used to normalize the Euclidean distance. v measures the similarity in aspect ratio between the predicted and ground truth boxes, and is calculated as follows:

[0189]

[0190] Where w and h represent the width and height of the prediction box, w gt and h gt This represents the width and height of the true bounding box, arctan represents the arctangent function, π represents pi, and τ is the adaptation weight coefficient, which is calculated as follows:

[0191]

[0192] Where v is used to measure the similarity of the aspect ratio between the predicted bounding box and the ground truth bounding box;

[0193] Let v represent the partial derivative; the partial derivatives of v with respect to the two variables w and h are calculated as follows:

[0194]

[0195] Where v is used to measure the similarity of the aspect ratio between the predicted bounding box and the ground truth bounding box. The partial derivative is represented by ; w and h represent the width and height of the predicted bounding box, and the corresponding w... gt and h gt represents the width and height of the true bounding box, arctan represents the arctangent function, and ∈ is a very small smoothing constant used to prevent calculation errors caused by a zero denominator;

[0196] Finally, we need to consider the method for calculating the classification loss. Since the targets detected and identified are mostly small targets, we will consider using DiceLoss here. Unlike traditional cross-entropy, DiceLoss focuses more on the matching of the overall structure rather than the accumulation of errors pixel by pixel. Unlike objects in ImageNet that have rich shape and texture features, the classification of text fields on the instrument usually depends on the subtle positional differences between characters and the overall arrangement pattern of the string. Since text fields consist of multiple characters with small trajectory differences (e.g., the number "1" and the letter "l"), problems such as blurred character edges and stroke adhesion significantly increase classification difficulty when the camera distance is far or the image resolution is low. Furthermore, complex factors such as the tilt angle of the instrument image, uneven lighting, and mechanical wear can cause localized reflections or blurring, further reducing the distinguishability of the text fields. To address these issues, during model training, a significantly higher loss weight is applied to misclassifications of low-frequency samples (rare categories) than to high-frequency samples (common categories). This forces the model to strengthen feature learning from sparse data and alleviates model bias caused by uneven category distribution. Therefore, an attribute-weighted loss function is specifically designed. loss Furthermore, to address the potential text region orientation flipping issue after perspective transformation, we designed a deep learning-based text orientation discrimination model, such as... Figure 4As shown, this model uses a lightweight convolutional neural network to construct a binary classification system, aiming to accurately distinguish between the standard orientation (0°) and the inverted state (180°) of text regions. Specifically, the input image first undergoes geometric normalization preprocessing, uniformly scaling the text region to a normalized dimension of 128×48 pixels. The network structure includes four feature extraction modules, with the third-to-last layer configured as a 4096-dimensional fully connected layer for advanced feature fusion. Subsequently, the dimension is reduced to a 26-dimensional hidden layer for feature compression. Finally, a probability distribution is generated through a dual-neuron output layer combined with the Softmax function. To address the class imbalance problem caused by interference factors such as extreme lighting conditions, large tilt angle shooting, and long-distance imaging, a DiceLoss function based on an adaptive weight adjustment mechanism is introduced.

[0197]

[0198] Among them, f α f represents the class balancing factor, responsible for balancing the class distribution. ο f represents the target scaling factor; δ This represents the attribute balance factor, which is specifically designed to handle complex image attributes in industrial scenarios. This indicates reducing the loss weight factor for easily classified samples; Dice loss The loss function used in image segmentation and classification tasks is calculated as follows:

[0199]

[0200] Among them, y n Indicates the real label, p n This refers to the model output calculated using the sigmoid function. ∈ is a very small smoothing constant used to prevent calculation errors caused by a zero denominator.

[0201] f α =α·y n +(1-α)·(1-y n )

[0202] f ο =ο·y n +(1-ο)·(1-y n )

[0203]

[0204]

[0205] Where α and For hyperparameters, ο represents the bounding box size factor; ο controls the target scale factor f. οFor large targets, a larger value will significantly reduce the loss weight of large targets, thus relatively amplifying the focus on small targets. Finally, a new factor was designed to address complex factors such as the tilt angle of the instrument image, uneven illumination, and mechanical wear. The samples are classified and labeled into one or more attribute classes, including tilt angle, uneven lighting, mechanical wear, etc. p represents the proportion of samples of class k; t The confidence score of the model for the correct category is calculated as follows:

[0206] p t =y n ·p n +(1-y n )·(1-p n )

[0207]

[0208] Among them, h min and w min This indicates the minimum height and width to be set, w gt and h gt This represents the width and height of the actual frame.

[0209] The optical instrument character detection module includes:

[0210] Recent research indicates that the SwinTransformer and EfficientNet architectures demonstrate superior performance and efficiency in text detection tasks when selecting the base network architecture. SwinTransformer, through its hierarchical VisionTransformer structure, better captures long-range dependencies in images, significantly improving text detection performance in complex scenes while reducing computational load. EfficientNet, however, is less adaptable to handling elongated targets; its convolutional architecture struggles to effectively capture long-range dependencies, leading to decreased detection accuracy. Therefore, considering both model performance and computational efficiency, the DB algorithm, combined with SwinTransformer, not only handles feature extraction but also enhances the detection capability for text at different scales by introducing attention mechanisms and multi-scale feature fusion techniques. The specific computation process is as follows:

[0211] P=σ(F(I))

[0212]

[0213] F(I)=Conv(SwinTransformer(I))

[0214] Where P is the generated probability map, I represents the input image, F(I) represents the final feature map, σ() represents the sigmoid function, M represents the binary map, exp represents the exponential operation with the natural constant e as the base, -λ represents a scaling factor that controls the smoothness of the function, and T represents the learnable threshold map. Represents the loss function. To predict the loss between the probability map P and the true label, μ and ξ are hyperparameters. It is the loss that predicts the relationship between the binary image M and the true label. It is a regularization term for the threshold map T. Conv() indicates convolutional layer processing, and SwinTransformer() indicates SwinTransformer network processing.

[0215] S5: Proportionally expand the borders of each text region detected in S4 (Padding).

[0216] Specifically, including:

[0217] Because the acquired images may be tilted, a text frame based on a specific format is introduced in the second stage. The borders of the detected text regions are expanded according to the following formula: Figure 5 As shown, to prevent characters from being missing;

[0218] x′ min =x min -x l ×(Δy)×d l

[0219] y′ min =y min -y t ×(Δy)×d t

[0220] x′ max =x max +x r ×(Δy)×d r

[0221] y′ max =y max +y d ×(Δy)×d d

[0222] Δy=y max -y min

[0223] Among them, (x min ,y min ),(x max ,y max() represents the coordinates of the top left and bottom right corners of the border; x l and x r y represents the left and right expansion scaling factor. t and y d d represents the scaling factor for vertical expansion. l and d r d represents the left and right expansion direction factor. t and d d This represents the vertical expansion direction factor; the values ​​of these parameters are adjusted according to the stage two target detection model; Δy represents the proportion of the expanded object, while (x′) represents the vertical expansion direction factor. min ,y′ min ),(x′ max ,y′ max The ) symbol is used to specify the position of the extended object; the filling process is as follows: Figure 4 As shown.

[0224] S6, input the reconstructed new image into a separate optical character recognition (OCR) system;

[0225] Specifically, this includes inputting the reconstructed image into a separate optical character recognition (OCR) system to identify and display optical characters, labels on optical numerical instruments, manufacturer names, models, readings, and units.

[0226] S7, during the identification process, handles any possible anomalies;

[0227] Specifically, in object detection tasks, the overlapping bounding boxes problem refers to the phenomenon where the model generates multiple highly overlapping predicted boxes for the same object, leading to redundant or misclassified results. The core challenge of this problem lies in how to select the most accurate and non-redundant detection result from a large number of candidate boxes. During project experiments, it was found that overlapping bounding boxes occur with a small probability during object detection. Therefore, an overlapping bounding box correction module was designed to solve this problem, specifically including the following steps:

[0228] Step 1: Input the original image I and detect the target boxes in the image to obtain the target box set B;

[0229] Step 2: Initialize the program; if |B|≥2, then for the target box B... i B j ∈B performs the following operation;

[0230] Among them, B i and B j For elements in set B;

[0231] Calculate B i and Bj The overlap is calculated using the following formula:

[0232]

[0233] Among them, IoU ij B i and B j The intersection-union ratio, Area(B) i ∩B j ) represents element B i and B j The intersection of Area(B) i ∪B j ) represents element B i and B j The union of the sets;

[0234] If IoU ij >T overlap Then calculate the center distance d ij The calculation formula is as follows:

[0235] d ij =‖Center(B i )-Center(B j )‖2

[0236] Where, d ij Center(B) represents the distance between centers. i ) and Center (B j ) represents element B i and B j The center coordinates, |||2 is used for center distance calculation, T overlap The threshold value for the overlapping portion of the target bounding box;

[0237] If d ij <T distance Then merge element B i and B j A new target box B is obtained. new The calculation formula is as follows:

[0238] B new =Merge(B i B j )

[0239] Among them, B new Merge() indicates merging newly generated target boxes, and T represents the merge operation. distance The threshold value for the set center distance;

[0240] And update set B, using the following formula:

[0241] B = {B i B j}∪{B new}

[0242] Step 3: Detect and merge set B to obtain a new target set B merged ;

[0243] Step 4: Output a new set of bounding boxes B merged .

[0244] S8 outputs structured text information, completing the end-to-end recognition process from raw image to data parsing.

[0245] To evaluate the performance of the proposed model, we will conduct comparative experiments, comparing it with the following seven deep learning-based detection and recognition methods:

[0246] 1. CTPN: This method combines convolutional neural networks with bidirectional RNNs to transform text detection into a sequence labeling problem. It proposes candidate regions for text lines by connecting vertical anchors. This method performs well in horizontal text detection tasks and is one of the early representative methods for text detection in deep learning scenarios.

[0247] 2. EAST: It adopts a fully convolutional network structure to predict the geometric parameters and scores of text boxes end-to-end. It supports rotated box detection. This method has extremely high detection speed while ensuring high accuracy, and is suitable for multi-directional text detection scenarios.

[0248] 3. PAN: Proposes the PixelAggregation mechanism, which takes lightweight structure and boundary-guided learning as its core, to improve detection speed while maintaining high accuracy, and is suitable for real-time text detection tasks.

[0249] 4. ABCNet2: Based on ABCNet, it further optimizes the method by using bounding box prediction of text instances. It reconstructs the quadrilateral of the text instance from the character bounding boxes of the text instance. At the same time, it introduces a character-to-word correction network to improve the detection accuracy and robustness of curved text.

[0250] 5. EK-Net: An innovative method for scene text detection, designed to solve the accuracy problems of existing detection methods when dealing with multi-scale, multi-directional and curved text in complex scenes, especially the "shrinking kernel" problem common in existing kernel-based methods, which is addressed by proposing an improvement scheme.

[0251] 6. RCNet: It introduces a fine-tuning correction module on top of the basic text detection network. First, it uses a feature extraction network to obtain the feature representation of the image and initially locates the text candidate region. Then, it uses a correction network to fine-tune these candidate regions, optimize the position and shape of the text box, and can effectively handle text in various complex scenarios.

[0252] 7. DPText-DETR: This is a Transformer-based scene text detection method. It adopts the idea of ​​object detection and directly predicts the bounding boxes of text instances. By leveraging the powerful modeling capabilities of Transformer, it can effectively capture long-distance dependencies and contextual information of text, thereby improving the detection accuracy of text in complex scenes.

[0253] 2. Model Comparison Test

[0254] A comparison was made with the state-of-the-art methods, including CTPN, EAST, PAN, ABCNet2, EK-Net, RCNet, and DPText-DETR.

[0255] As can be clearly seen from Table 1, the CTPN and EAST metrics are all below 80%, with CTPN's recall rate at only 47.6% and its F-score below 54%. In contrast, the PAN model shows improvements in precision (80.1%) and F-score (78.4%), achieving a frame rate of 23.1 FPS, demonstrating a certain advantage in real-time performance. Figure 6 As shown, with technological advancements, next-generation models such as ABCNetv2, EK-Net, and DPText-DETR have achieved significant breakthroughs in detection accuracy. DPText-DETR, in particular, excels in precision (94%) and F-score (93.6%), but its frame rate is only 16.9 FPS. Notably, EK-Net maintains high detection accuracy (F-score 91.7%) while achieving a frame rate of 37.8 FPS, demonstrating a significant advantage in efficiency. The model proposed in this paper, based on the SwinTransformer architecture, achieves state-of-the-art results across all three evaluation metrics: recall (92.4%), precision (94.8%), and F-score (93.6%). Compared to DPText-DETR, our model improves recall, precision, and F-score by 4.8%, 4.4%, and 4.6%, respectively, and also increases the frame rate by 8.8 frames per second. Compared to EK-Net, it leads in precision and F-score by 9.3% and 6.3%, respectively.

[0256] Table 1: Performance Comparison Results of Models on Industrial Datasets

[0257] Method Architecture Precision Recall Detection accuracy Frame rate CTPN VGG16 62.2 47.6 53.9 7.1 EAST ResNet50 79.5 70.3 74.6 13.2 PAN ResNet50 80.1 76.7 78.4 23.1 ABCNetv2 ResNet50 93 91.5 92.2 8.7 EK-Net ResNet50 91.5 92 91.7 37.8 DPText-DETR ResNet50 94 93.2 93.6 16.9 RCNet ResNet50 90.5 91 90.7 13.6 Ours SwinTransformer 92.4 94.8 93.6 25.7

[0258] 3. Ablation test

[0259] To gain a deeper understanding of the contributions and interactions of the various modules (CNSel, Attribute-Weighted Loss, Angle Transformation, and Overlapping Box Correction) proposed in this paper to model performance, we conducted a series of comprehensive ablation experiments. By systematically enabling and disabling different combinations of modules, we aimed to quantify the independent effects of each module and the overall benefits of their combination. The experimental results are summarized in Table 2. A total of 16 different module combinations were examined in this experiment.

[0260] As shown in Table 2, without any additional modules, the model's F1 score is 70.5%, R is 70.0%, and P is 71.0%. Adding Attribute-Weighted Loss to the baseline (F = 70.5%) alone improves the F1 score to 78.6%, meaning the Attribute-Weighted Loss module alone contributes 8.1 percentage points, making it the largest contributor among all modules. When all four modules are enabled, the model reaches its highest performance, with the F1 score jumping to 93.6%, and R and P reaching 92.4% and 94.8%, respectively. This indicates that all proposed modules positively contribute to improving model performance, and there is good compatibility and cumulative effect among them. From the baseline to the full module configuration, the F1 score improves by 23.1 percentage points, validating the effectiveness of combining the modules.

[0261] These results further demonstrate that the modules can synergistically enhance each other and significantly improve the model's performance. At the same time, it can be observed that, with the same number of modules, combinations that include more influential modules usually achieve better performance, which further reinforces the impact of modules on model performance.

[0262] Table 2: Ablation Experiment Results

[0263]

[0264] 4. Performance Impact Analysis of CNSel (Channel Normalized Selection)

[0265] like Figure 7As shown, the experimental results verify the effectiveness of the channel normalization selection module under different lighting conditions. Under known lighting conditions, both Ours and Ours-CNSel can achieve relatively ideal recognition results. However, under unknown lighting conditions, especially in brighter or darker scenes, the robustness of Ours decreases significantly, manifested as blurred digit display and insufficient recognition stability. In contrast, Ours-CNSel can maintain the clarity and stability of digits under various lighting changes. This indicates that the channel normalization selection module can effectively improve the generalization ability and robustness of the model under complex lighting conditions by selectively normalizing and enhancing lighting-invariant features, thus better adapting to the application needs of actual industrial environments.

[0266] 5. Performance Impact Analysis of Attribute-Weighted Loss (Loss Function)

[0267] To clearly verify the effectiveness of our proposed Attribute-Weighted Loss function, we first constructed a baseline model as a comparison. This baseline model uses the exact same network architecture as our final model, namely an advanced and efficient real-time detector architecture (YOLOv10). The only difference is that the baseline model uses a standard loss function in the classification part, while our proposed full model replaces it with Attribute-Weighted Loss. The experiments evaluate the model's performance under different precision / recall tradeoffs by adjusting the confidence thresholds (0.5 and 0.7).

[0268] As shown in Tables 3 and 4, our model consistently outperforms the baseline, especially at the more stringent confidence threshold of 0.7, where its superiority is amplified. The baseline's false negative rate on some fields soars to 0.241, which is 0.114 higher than our model. This strongly demonstrates that, compared to the baseline, which shows a significant performance degradation under stringent conditions, our proposed IMR-Net model has stronger robustness and superior overall performance, making it better suited for demanding real-world applications.

[0269] Table 3: Tag object detection results when the confidence threshold is 0.5

[0270]

[0271] Table 4: Tag object detection results when the confidence threshold is 0.7

[0272]

[0273] 6. Analysis of the impact of scene text orientation detection performance

[0274] To evaluate the performance of our proposed method, a benchmark set was specifically established, covering various image poses including canonical orientation and 180-degree flip. Experimental data (see Table 5) show that the model performs excellently in the text orientation discrimination task, achieving an accuracy of 98.56%. Compared to the orientation discrimination model used by PP-OCR, our proposed solution (details in Section 3.4) demonstrates stronger competitiveness, achieving an accuracy metric 0.73 percentage points higher.

[0275] Table 5: Comparison with PP-OCRv3

[0276] Method Input resolution Accuracy Model parameter size PP-OCRv3 3*32*128 97.83％ 2.37Mb Ours 3*32*160 98.56％ 3.24Mb

[0277] 7. Performance Impact Analysis of Deflection Angle Quantization

[0278] The performance of deflection angle quantization was tested using tilted images. The results are as follows: Figure 8 As shown, this method is less accurate than for images with moderate skew angles when dealing with extremely small or large skew angles. Specifically, when the skew angle is very small, annotators find it difficult to accurately draw rectangles with irregular four points. This naturally leads to a larger range of deviations and relative errors. While the calculated deviation may be slightly higher when the skew angle is very large, the relative error remains low. This smaller relative error is considered acceptable when dealing with these larger skew angles.

[0279] 8. Analysis of the impact of end-to-end optical character recognition performance

[0280] To construct an end-to-end optical character meter recognition system, we integrated the various functional modules in the proposed architecture. We validated the effectiveness of key components through a series of ablation experiments, specifically the filling strategy and our proposed auxiliary module—rotation angle calculation. As shown in Table 6, the experimental results clearly demonstrate that the filling strategy significantly improves the accuracy of all recognition fields. For example, the recognition accuracy in the "Type" field is improved by 10.2 percentage points. Furthermore, the experimental data in Table 7 verify that the auxiliary module can enhance the model's robustness to recognition of images tilted at arbitrary angles.

[0281] Table 6: Performance Comparison Regarding the Effectiveness of Filling Strategies

[0282]

[0283] Table 7: Performance Comparison of Adding Tilt Angle Calculation in End-to-End Instrument Tag Recognition

[0284]

[0285] 9. Analysis of the impact of different hyperparameters on the model

[0286] In visual inspection tasks, we conducted systematic experimental analysis on the values ​​of hyperparameters α and γ, such as... Figure 9 As shown in the figure, the results indicate that both α and γ exhibit a significant unimodal distribution characteristic in terms of detection accuracy. This means that within a certain range, both α and γ can significantly improve model performance, while excessively small or large values ​​will lead to a decrease in detection accuracy. Specifically, as α increases, the detection accuracy reaches its optimum (80.38%) at α = 0.35, then rapidly decreases and tends to stabilize. This suggests that α has an optimal point when adjusting the balance between positive and negative samples or different loss terms. On the other hand, the change in γ has a more sensitive impact on model performance, achieving the highest accuracy (83.33%) at γ = 1.7. Further increases will excessively weaken the contribution of easy samples, leading to an overall performance decline. In summary, the reasonable selection of α = 0.35 and γ = 1.7 can effectively improve the robustness and accuracy of the detection system in complex scenes, verifying the crucial role of adjusting the weights of easy and difficult samples and balancing the loss terms in visual detection.

[0287] 10. Visual analysis of detection results

[0288] To verify the effectiveness of the proposed Attribute-Weighted Loss function, simple samples were selected for the experiment. Figure 10 Medium (a) and difficult samples Figure 10 Compare the two models in (b). To ensure fairness, the two models are identical except for the loss function.

[0289] As shown in the figure, the red and yellow rectangles represent the detection results of the models using Attribute-Weighted Loss and the standard Baseline loss function, respectively. On simple samples, both methods successfully detect all targets. However, on difficult samples with uneven lighting and tilted angles, the baseline model can only identify the instrument readings, missing other crucial information. In contrast, the model using Attribute-Weighted Loss successfully detects all labels, demonstrating that our proposed loss function significantly improves the model's detection performance and robustness on difficult samples.

Claims

1. A robust end-to-end method for optical character recognition of industrial instruments under complex working conditions, characterized in that, Includes the following steps: S1: Preprocess the images in the dataset; S2: Input the preprocessed image into the channel normalization selection module for further processing; S3: Detect the tilt angle of the processed image and determine whether its tilt angle is greater than the preset threshold. If the angle is less than or equal to the threshold, the image will proceed directly to the next step; if it is greater than the threshold, the image will be geometrically transformed and corrected before proceeding to the next step. S4: Input the image obtained in S3 into the lightweight object detection model for detection; S5: Expand and reorganize the borders of each text region detected in S4 proportionally; S6: Input the expanded and reconstructed new image into the optical character recognition system; S7: Handle any abnormal situations that occur during the identification process; S8: Outputs structured text information, completing the end-to-end recognition process from raw image to data parsing.

2. The robust end-to-end method for optical character recognition of industrial instruments under complex working conditions according to claim 1, characterized in that, The channel normalization selection module includes: Dynamic adaptive instance light normalization module and dynamic adaptive gating module.

3. The robust end-to-end method for optical character recognition of industrial instruments under complex working conditions according to claim 2, characterized in that, The dynamic adaptive instance light normalization module includes: The process of the dynamic adaptive instance light normalization module is as follows: Where ⊙ denotes channel-wise multiplication, x represents a 3D feature map, x′ represents the feature map x after DAIN normalization, DAIN() represents DAIN normalization, and γ, This represents non-fixed learnable parameters that are dynamically generated from input features. Let δ represent a set of real numbers on channel C; δ is a very small positive real number, max() represents taking the maximum value, and σ c (x) represents the channel standard deviation of the 3D feature map x; μ c (x) represents the channel mean of the 3D feature map x, which is calculated as follows: Where H represents the height of the 3D feature map x, W represents the width of the 3D feature map x, and x i,j,c This represents a specific numerical value in the three-dimensional feature map x; σ c (x) represents the channel standard deviation of the 3D feature map x, which is calculated as follows: Each channel c∈{1,2,...,C} is computed independently to ensure that the lighting differences between instances are suppressed, and C represents the number of channels in the 3D feature map x. The calculation methods for γ and β are as follows: γ=W γ ·GAP(x),β=W β ·GAP(x) Among them, W γ , It is a learnable weight matrix, which reduces redundancy through sparse initialization; Let C be a set of C rows and C columns of real numbers. GAP() represents global average pooling. DAIN normalization can reduce the lighting differences between instances, so the feature x′ normalized by DAIN has robust properties that are independent of lighting conditions.

4. A robust end-to-end method for optical character recognition of industrial instruments under complex working conditions, as described in claim 2, is characterized in that... The dynamic adaptive gating module includes: The processing flow of the dynamic adaptive gating module is as follows: x n+1 =γ(g)⊙x n ′+(1-γ(g))⊙x n Where γ(g) is the channel attention weight, ⊙ represents channel-wise multiplication, and x n This represents the original feature map input to the channel normalization selection module, x. n ′ represents x n The feature map after normalization, x n+1 This represents the generated target feature map; The channel attention weight γ(g) is calculated as follows: γ(g)=σ(W'·GAP(g)) Where σ() is the Sigmoid function, g represents a binary indicator along the channel dimension, W' represents a weight matrix, and GAP() represents global average pooling. Therefore, the generated target feature map x n+1 By eliminating the effects of illumination, a constant representation is obtained, along with unchanged channels, while preserving crucial information to support accurate reconstruction. Specifically, the gating operation should be differentiable and able to limit the output to 0 or 1 for channel selection. The gating module is designed as shown below, represented as follows: g=G(α x )=σ(ω·α x ) Where ω is the scaling factor, α x It is an intermediate vector generated from the input features, where σ() is the Sigmoid function and G() represents a gating function. Specifically, firstly, in order to construct the intermediate vector α associated with the input features... x The spatial dimension of the features is compressed to a single pixel scale through global average pooling, and then the final representation is obtained through a linear transformation. Finally, a scaling factor ω is introduced into the gating function. When α x >>0, resulting in g = G(α) x )≈1; when α x << 0 yields g = G(α) x Therefore, the designed function G(α) ≈ 0; x Able to convert any α x Effectively mapped to the interval [0,1].

5. A robust end-to-end method for optical character recognition of industrial instruments under complex working conditions according to claim 1, characterized in that, The image tilt angle detection includes: Step 1: Input the original image I and obtain the low threshold T low and high threshold T high Used to determine whether a pixel belongs to an edge; obtain the Hough parameter ρ res =1,θ res =1°, accumulator threshold T acc Minimum line segment length L min and maximum gap G max ; Image I is obtained by grayscale conversion of the original image I. gray ; Step 2: Initialize the Hough accumulator and extract image I. gray Each edge point (x i ,y i ); For θ∈[0°,180°) with ρ res and θ res Traverse for step size and angle; The formula for calculating ρ is as follows: p=x i cosθ+y i sinth Where, x i and y i The x and y coordinates of the edge point are represented, where θ represents the angle of the edge point and ρ represents the radial distance in the polar coordinate system. Secondly, the input to the Hough accumulator yields a candidate straight-line threshold A(ρ). j ,θ j ); Step 3: Extract candidate lines, if A(ρ j ,θ j )≥T acc Record (ρ) j ,θ j ); Where, A(ρ) j ,θ j ) represents the candidate line threshold, T acc ρ represents the accumulator threshold. j and θ j These represent the step size of the Hough accumulator in the distance and angle dimensions, respectively; Step 4: Determine the length and gap of each candidate line. If the line segment length is ≥ L... min And the gap ≤ G max If so, then retain it; Step 5: Construct an angle set Θ = {θ} j };Θ filtered ={θ∣-45°<θ<45° or 135°<θ<180°}; θ skew =Median(Θ) filtered ); Where Θ represents an angle set used to store the angles of all detected candidate lines; Θ filtered This represents the filtered set of angles; Median() means taking the median, θ skew Indicates the tilt angle; Step 6: Calculate the tilt angle θ skew And output it.

6. A robust end-to-end method for optical character recognition of industrial instruments under complex working conditions according to claim 1, characterized in that, The lightweight target detection model includes: The module includes a character area detection module for optical instruments and meters, a structured text module, and a character detection module for optical instruments and meters.

7. A robust end-to-end method for optical character recognition of industrial instruments under complex working conditions, as described in claim 6, is characterized in that... The optical instrument character area detection module includes: The localization loss function is optimized to address challenges in industrial scenarios, such as image tilt and target scale differences caused by varying viewing angles and long-distance shooting, as well as character blurring, glare, and stroke adhesion caused by uneven lighting and equipment aging. Considering that the CIoU loss function can take into account the aspect ratio of the detection box, it is used as the core optimization criterion for image annotation target detection tasks, as shown below: in, CIoU represents the loss function, and IoU represents the intersection-over-union ratio. This represents the penalty term in the CIoU loss function, used to measure the positional and shape differences between the predicted and ground truth boxes. Its calculation method is as follows: Where τ is the adaptive weight coefficient, ρ 2 (b,b gt ) represents the squared Euclidean distance between the predicted bounding box and the ground truth bounding box, used to measure the difference in their center point positions; d 2 represents the square of the diagonal length of the smallest closure covering both the predicted and ground truth boxes, used to normalize the Euclidean distance. v measures the similarity in aspect ratio between the predicted and ground truth boxes, and is calculated as follows: Where w and h represent the width and height of the prediction box, w gt and h gt These represent the width and height of the true bounding box, arctan represents the arctangent function, and π represents pi; τ is calculated as follows: The partial derivatives of v with respect to the two variables w and h are calculated as follows: in, Let r represent the partial derivative, r be the calculation correction term, and ∈ be a very small smoothing constant used to prevent calculation errors caused by a zero denominator; Finally, a targeted attribute-weighted loss was designed, and a suitable loss function, Attribute-Weighted, was developed. loss : Among them, f α f represents the class balancing factor, responsible for balancing the class distribution. ο f represents the target scaling factor; δ This represents the attribute balance factor, which is specifically designed to handle complex image attributes in industrial scenarios. This indicates reducing the loss weight factor for easily classified samples; Dice loss The loss function used in image segmentation and classification tasks is calculated as follows: Among them, y n Indicates the real label, p n This refers to the model output calculated using the sigmoid function; f α =α·y n +(1-a)·(1-y n ) f ο =o·y n +(1-o)·(1-y n ) Where α and For hyperparameters, ο represents the bounding box size factor; ο controls the target scale factor f. ο The weighting suppression strength for large targets is determined by a larger value, which will significantly reduce the loss weight of large targets, thus relatively amplifying the focus on small targets. Finally, a new factor is designed. Classify and label samples as one or more attribute classes; p represents the proportion of samples of class k; t The confidence score of the model for the correct category is calculated as follows: p t =y n ·p n +(1-y n )·(1-p n ) The calculation method for ο is as follows: Among them, h min and w min This indicates the minimum height and width to be set.

8. A robust end-to-end method for optical character recognition of industrial instruments under complex working conditions, as described in claim 6, is characterized in that... The structured text module includes: Because the acquired image may be tilted, a text frame based on a specific format is introduced. The borders of the detected text regions are expanded according to the following formula: x′ min =x min -x l ×(Δy)×d l y′ min y min -y t ×(Δy)×d t x′ max =x max +x r ×(Δy)×d r y′ max y max y d ×(Δy)×d d Δy = y max -y min Among them, (x min ,y min ),(x max ,y max () represents the coordinates of the top left and bottom right corners of the border; x l and x r y represents the left and right expansion scaling factor. t and y d d represents the scaling factor for vertical expansion. l and d r d represents the left and right expansion direction factor. t and d d This represents the vertical expansion direction factor; the values ​​of these parameters are adjusted according to the stage two target detection model; Δy is the proportion of the expanded object, while (x′) represents the vertical expansion direction factor. min ,y′ min ),(x′ max ,y′ max The ) symbol is used to specify the location of the extended object.

9. A robust end-to-end method for optical character recognition of industrial instruments under complex working conditions, as described in claim 6, is characterized in that... The optical instrument character detection module includes: The optical instrument character detection module improves its ability to detect text at different scales by introducing an attention mechanism and multi-scale feature fusion technology; the specific calculation process is as follows: P=σ(F(I)) F(I)=Conv(SwinTransformer(I)) Where P is the generated probability map, I represents the input image, F(I) represents the final feature map, σ() represents the sigmoid function, M represents the binary map, exp represents the exponential operation with the natural constant e as the base, -λ represents a scaling factor that controls the smoothness of the function, and T represents the learnable threshold map. Represents the loss function. To predict the loss between the probability map P and the true label, μ and ξ are hyperparameters. It is the loss that predicts the relationship between the binary image M and the true label. It is a regularization term for the threshold map T. Conv() indicates convolutional layer processing, and SwinTransformer() indicates SwinTransformer network processing.

10. A robust end-to-end method for optical character recognition of industrial instruments under complex working conditions according to claim 1, characterized in that, S7 includes: There is a small probability of overlapping bounding boxes occurring during object detection; therefore, an overlapping bounding box correction module was designed to solve this problem, including: Step 1: Input the original image I and detect the target boxes in the image to obtain the target box set B; Step 2: Initialize the program; if |B|≥2, then for the target box B... i B j ∈B performs the following operation; Among them, B i and B j For elements in set B; Calculate B i and B j The overlap is calculated using the following formula: Among them, IoU ij B i and B j The intersection-union ratio, Area(B) i ∩B j ) represents element B i and B j The intersection of Area(B) i ∪B j ) represents element B i and B j The union of the parts; If IoU ij >T overlap Then calculate the center distance d ij The calculation formula is as follows: d ij =‖Center(B i )-Center(B j )‖2 Where, d ij Center(B) represents the distance between centers. i ) and Center (B j ) represents element B i and B j The center coordinates, |||2 is used for center distance calculation, T overlap The threshold value for the overlapping portion of the target bounding box; If d ij <T distance Then merge element B i and B j A new target box B is obtained. new The calculation formula is as follows: B new =Merge(B i ,B j ) Among them, B new Merge() indicates merging newly generated target boxes, and T represents the merge operation. distance The threshold value for the set center distance; And update set B, using the following formula: B={B i ,B j }∪{B new } Step 3: Detect and merge set B to obtain a new target set B merged ; Step 4: Output a new set of bounding boxes B merged .

Images