A method for digital meter reading based on low-quality images

Abstract

The application discloses a kind of digital instrument reading detection methods based on low-quality image, belong to intelligent detection and identification technical field of industrial equipment.It first acquires digital instrument image dataset, and is preprocessed;Based on feature extraction network, multi-scale feature map is extracted, and after end of down-sampling, shallow information containing more position information is directly transmitted upward using the enhancement path from bottom to top in FPN module, and the utilization of shallow position information is improved.Fusion factor is added in adjacent layers of FPN to promote shallow centralized learning fuzzy target, and then the robustness of segmentation network is improved.The set approximate binary function is a binary quantization function that can automatically adjust and approximate the standard binary function during training.The application solves the robustness of image blur caused by defocus and shaking, and the difficulty in detecting instrument reading area caused by shadow and foreign object obstruction.In the detection of low-quality image, the detection performance of the application is better.

Description

Technical Field

[0001] This invention belongs to the field of intelligent detection and identification technology of industrial equipment, and specifically relates to a method for detecting digital instrument readings based on low-quality images. Background Technology

[0002] The past decade has witnessed the rapid development of computer vision in many fields, with deep learning-based image detection and recognition technologies increasingly being applied to intelligent industrial manufacturing. Digital instrument reading recognition is one such widely used task.

[0003] As the first step in digital instrument reading recognition, the goal of reading detection is to locate the bounding box or boundary region of the instrument's readings. This remains a challenging task in practical applications. In real-world scenarios, the images input to the algorithm may suffer from focusing failures, image blurring due to movement, or obstructions from shadows or foreign objects, making reading area detection difficult. Furthermore, most current scene text detection methods are not optimized for low-quality images, resulting in less than ideal detection performance.

[0004] Pixel-level segmentation-based text detection methods have gained popularity in recent years because they can generate prediction maps, similar to image segmentation methods, indicating whether each pixel in the original image belongs to any text instance. However, most segmentation-based detection methods (such as PSENet and SAE) require complex post-processing to group pixel-level predictions into detected text instances, which increases computational overhead during inference and impacts real-time processing efficiency. Researchers have proposed a differentiable approximate binarization function, which is trained together with a segmentation network to reduce post-processing computational overhead and meet the real-time inference requirements of practical applications. However, the approximate binarization function of the differentiable binarization algorithm is actually a sigmoid function with a fixed curvature of 50, which does not always adapt well to various types of datasets. Summary of the Invention

[0005] This invention provides a digital instrument reading detection method based on low-quality images to solve the technical problems of image blurring caused by defocusing and shaking, and difficulty in detecting the reading area of ​​the instrument due to shadows and foreign objects.

[0006] The technical solution adopted in this invention is as follows:

[0007] A method for detecting digital instrument readings based on low-quality images, the method comprising:

[0008] Step S1: Acquire the images captured by the digital instrument and perform image preprocessing to obtain the image dataset of the digital instrument;

[0009] Step S2: Input the images in the image dataset into the feature extraction network for multi-scale feature extraction;

[0010] Step S3: The feature extraction network extracts multi-scale feature maps, which are then input into the feature pyramid network FPN module for feature fusion to obtain feature map F;

[0011] Step S4: Generate a probability map P and a threshold map T by predicting the feature map F;

[0012] Step S5: Input the probability map P and the threshold map T into the adaptive differentiable binarization module to calculate the approximate binary map.

[0013] Step S6: Obtain the text bounding box of the image acquired by the digital instrument through the bounding box generation module, and then generate the text bounding box from the approximate binary image. Extract the display area image.

[0014] Furthermore, in step S2, C is defined in an upward direction. i The feature map of the i-th layer is represented in a top-down order, according to the formula. Obtain the first fusion result of the i-th layer, where P M =C M M represents the number of scales extracted by the feature extraction network, and the first fusion result is only calculated up to the second layer; f chn f represents a 1×1 convolution operation used for channel matching. up This indicates a 2x upsampling operation used for size matching. This represents the fusion factor from layer (i+1) to layer (i).

[0015] The second fusion result N is generated in a bottom-up order. i Where N2 = P2, the N of the i-th layer i After downsampling, the first fusion result P of the (i+1)th layer i+1 By performing element-wise addition, we obtain the second fusion result N of the (i+1)th layer. i+1 .

[0016] Furthermore, in step S2, a ResNet-18 or ResNet-50 backbone network is used as the feature extraction network.

[0017] Furthermore, in step S2, the feature extraction network extracts feature maps at four scales, with each feature map having a size of 1 / 2, 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the input image of the feature extraction network.

[0018] Furthermore, in step S2, the FPN module uses a bottom-up path enhancement structure to perform feature fusion on multi-scale feature maps.

[0019] Furthermore, in step S5, the approximate binary image... Specifically:

[0020]

[0021] Where δ represents the similarity factor, δ∈(0,1), P i,j T i,j Representing approximate binary graphs The values ​​of the probability map P and the threshold map T at pixel (i,j).

[0022] Furthermore, in step S6, the probability map P or the approximate binary map is... The bounding box generation module obtains the text bounding box of the image acquired by the digital instrument.

[0023] The technical solution provided by this invention brings at least the following beneficial effects:

[0024] (1) It combines accuracy and real-time performance, and can meet the actual needs of industrial application environments.

[0025] (2) It can detect low-quality images generated under non-ideal imaging conditions.

[0026] (3) Better detection accuracy was achieved on general datasets and digital instrument reading datasets than previous methods. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 This is a flowchart of a digital instrument reading detection method based on low-quality images provided in an embodiment of the present invention;

[0029] Figure 2 This is a schematic diagram of the structure of the backbone network and FPN module in an embodiment of the present invention.

[0030] Figure 3 This is a schematic diagram of the feature pyramid module in an embodiment of the present invention.

[0031] Figure 4 This is a schematic diagram of a bottom-up path enhancement unit in an embodiment of the present invention.

[0032] Figure 5This is a schematic diagram comparing the standard binarized and approximate binarized function images in an embodiment of the present invention.

[0033] Figure 6 This is a schematic diagram of the function graph of the same approximate binary function under different curvatures in an embodiment of the present invention.

[0034] Figure 7 This is a schematic diagram of the function graphs of different approximate binary functions under the same curvature in an embodiment of the present invention.

[0035] Figure 8 This is a schematic diagram of the image acquired by the digital instrument in an embodiment of the present invention.

[0036] Figure 9 This is a schematic diagram of the processing results of the images acquired by the digital instrument in an embodiment of the present invention. Detailed Implementation

[0037] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0038] The first step in digital instrument reading recognition is the detection of the instrument's reading area. Then, reading recognition can be performed on the detected reading area, which is essentially image-based digital recognition processing. The accuracy of reading recognition is directly related to the correct detection of the reading area. Pixel-level segmentation methods are quite popular in scene text detection, as they offer good performance in both accuracy and efficiency, making them suitable for real-time scene text detection tasks in digital instrument reading recognition.

[0039] Features from different levels can be fused to form feature pyramids, which can then be applied to tasks such as object detection and image segmentation, demonstrating excellent performance in improving model performance. Feature pyramids have the characteristic of different resolutions at different scales, allowing targets of different sizes to have appropriate feature representations at their respective scales. By fusing multi-scale information, predictions can be made for targets of different sizes at different scales, thus significantly improving model performance.

[0040] There are generally two ways to construct a feature pyramid: one is to build the feature pyramid by generating layers of different resolutions through multiple downsampling operations, such as SSD (Single Shot MultiBox Detector), FPN (Feature Pyramid Networks), and YOLOv3. The other is to construct it by using multiple branches of dilated convolutions with different dilation rates, such as ASPP and RFP.

[0041] For post-construction processing, various methods have proposed different improvements. FPN improves the semantic information of lower-level feature maps by adding a top-down path to the pyramid. PANet, based on the idea of ​​shortening the information flow path and adding different branches to increase the information flow path, proposes adding a bottom-up path to the top-down path in FPN. NAS-FPN uses Neural Architecture Search (NAS) to combine and update the feature maps extracted by the backbone network, and uses the searched irregular topology as the pyramid structure to achieve better detection accuracy. BiFPN simplifies PANet, establishing a clean lateral connection path from low to high levels. CBNet uses composite connections between adjacent backbone networks to assemble multiple backbone networks, thus forming a powerful backbone network.

[0042] The standard binarization function (STE) is similar to the sign function, but unfortunately, its gradient is discontinuous and non-differentiable. This prevents backpropagation during training of convolutional neural networks, thus preventing the binarization step from being incorporated into the network training process. To address the non-differentiability of the STE gradient, a straight-through estimator (STE) is typically used to approximate the gradient. However, due to the significant gradient mismatch between the actual gradient of the STE and the actual gradient, backpropagation error can easily accumulate, causing the network training to deviate from the normal extreme points, resulting in insufficient network optimization and severely degrading performance. Therefore, researchers have proposed various carefully designed approximate binarization functions to alleviate the gradient mismatch problem in backpropagation.

[0043] Bi-Real Net provides a custom approximation binarization function, ApproxSign, to replace the sign function for gradient calculation during backpropagation. BNN+ proposes using a swish-sign function to approximate the sign function to obtain a better approximate gradient. The DSQ method proposes a DSQ function to simulate the rounding function, approximating the round function as closely as possible by introducing an adaptive similarity factor. DB Net directly uses a sigmoid function with fixed curvature to approximate the sign function, thus incorporating the binarization process into network training.

[0044] In practical applications, poor image quality often leads to suboptimal detection results. This invention proposes a Retina DB algorithm based on the DBNet network framework to address the issue of image blurring caused by defocusing and shaking, and difficulties in detecting display reading regions due to shadows and foreign object occlusion, thus demonstrating superior robustness. This invention utilizes a bottom-up enhancement path in the FPN module, directly passing shallow information containing more location information upwards after downsampling, improving the utilization rate of shallow location information. Furthermore, a fusion factor is added to adjacent layers of the FPN to drive shallow layers to learn blurred targets, thereby improving the robustness of the segmentation network. The approximate binarization function of Retina DB (Retina Differentiable Binarization) is modified to a binary quantization function that automatically adjusts and approximates the standard binarization function during training. In detecting low-quality images, this invention outperforms most existing methods.

[0045] like Figure 1 As shown, as one possible implementation, the digital instrument reading detection method based on low-quality images provided by this embodiment of the invention includes the following steps:

[0046] Step S1 involves acquiring the digital instrument dataset (i.e., acquiring images captured by the digital instrument) and performing data annotation and preprocessing operations (data augmentation, normalization, and formatting). Data annotation and data augmentation are used for training the network parameters.

[0047] Step S2: The preprocessed dataset is fed into the backbone network (a differentiable binarizable backbone network, preferably ResNet-18) to extract features, such as... Figure 2 As shown.

[0048] Among them, the differentiable binarization network refers to the improvement of the feature pyramid module and the (adaptive) differentiable binarization module based on the DB Net (Differentiable BinarizationNet) framework in this invention.

[0049] Step S3: Input the obtained feature map into the FPN module for feature fusion to obtain feature map F, such as... Figure 2 and Figure 3 As shown.

[0050] Step S4: Use the feature map F to predict the generation probability map P and the threshold map T.

[0051] Step S5: Input the probability map P and the threshold map T into the adaptive differentiable binarization module to calculate the approximate binary map.

[0052] Step S6: Obtain the text bounding box of the image acquired by the digital instrument through the bounding box generation module, and then generate the text bounding box from the approximate binary image. The image of the display area is extracted. Then, based on the image's digit recognition, the corresponding reading can be identified.

[0053] The network structures involved in the method of this invention include: a backbone network (feature extraction network), a feature fusion network (based on the FPN module to obtain feature map F), and a segmentation network (to obtain an approximate binary map). The relevant parameters of each network can be optimized based on learning and training.

[0054] Preferably, in step S3, the obtained feature map is input into the FPN module for feature fusion to obtain feature map F, specifically including:

[0055] Step S301, the bottom-up path enhancement structure of the FPN module is as follows:

[0056] Generally, higher layers of a network contain more semantic features, while lower layers contain more location features. A common practice in Faster Neural Networks (FPNs) is to perform upsampling after downsampling and then obtain information from the downsampled layers at the same level through lateral connections. The aim is to enhance the features of all layers by propagating the semantic information from higher layers back through this backpropagation.

[0057] Because low-level location information contains more edge and shape features, it is crucial for pixel-level segmentation. Therefore, a bottom-up path enhancement structure is introduced in the FPN module of DB Net to include more text location information in the feature map F, thereby improving the network's detection performance.

[0058] The FPN module in this embodiment of the invention is as follows: Figure 3 As shown, C1 to C5 are feature maps extracted from the backbone network, and their sizes are 1 / 2, 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the original image, respectively. Feature map O i By O i+1 After 2x upsampling, and C i The elements are added together. O5 and C5 are identical and have not undergone any processing. Feature maps N2 to N5 are obtained as follows: Figure 4 The bottom-up path enhancement unit shown is generated. Each unit is generated from a large-size feature map N. i and smaller P i+1 After fusion and connection, a new feature map N is generated. i+1 Each feature map N i Both require downsampling using a 3×3 kernel and a stride of 2, and then Pi+1 The feature maps obtained by downsampling are added element-wise to obtain N for feature map concatenation. i+1 It's important to note that P2 and N2 are the same and require no processing. During the feature fusion stage, the number of channels in the feature map remains constant, and a ReLU activation layer is added after each convolutional layer.

[0059] The number of channels in the feature maps N2-N5 obtained after feature fusion is reduced to 1 / 4 of their original size. Then, N5, N4, and N3 are upsampled by 8x, 4x, and 2x respectively. Finally, feature maps are concatenated to obtain the feature map F used for text detection. The size of feature map F is 1 / 4 of the input image, and the number of channels is the same as the feature map output by the backbone network. Figure 1 To.

[0060] In step S302, the FPN fusion factor is calculated as follows:

[0061] The higher layers of the FPN contain more large target features due to their smaller feature map size, while the lower layers have larger feature maps that can include more small target features. Since target characters in text detection tasks often appear as small objects in images, this invention aims to improve the network's ability to detect small targets by focusing on learning small targets in the lower layers. Therefore, a fusion factor σ is added to the feature fusion process of the FPN from top to bottom. This fusion factor controls the information passed from higher to lower layers, encouraging the shallower layers to focus on learning small objects, thus adapting the FPN to the detection of small targets.

[0062] With the fusion factor σ, adjacent layers P i and P i+1 The feature fusion process between them can be expressed by the following formula:

[0063]

[0064] Among them, f chn It is a 1×1 convolution operation for channel matching, f up It is a 2x upsampling operation for size matching. Representative from P i+1 Layer to P i Fusion factor for layer feature fusion.

[0065] Preferably, in step S5, the probability map P and the threshold map T are fed into the adaptive differentiable binarization module to calculate an approximate binary map. The specific implementation is as follows:

[0066] In step S501, the approximate binarization function is:

[0067] DB Net improves model performance by incorporating a segmentation threshold into training by replacing the standard binarization function with an approximate binarization function. It posits that the closer the approximate binarization function (DB) is to the standard binarization function (SB), the better the model's detection performance. Figure 5 As shown in 6 and 7.

[0068] Standard binarization typically uses the sign function as the binarization function.

[0069]

[0070] Using DB Net's approximate binarization function For example,

[0071]

[0072] Different curvatures k of the function directly affect the approximation degree of binarization. Figure 6 The results show that as the value of k gradually increases (k = 1, 10, 50, 100), the graph of the function becomes closer to the standard binarized function. Furthermore, different approximate binarized functions also differ in their degree of approximation to the standard binarized function at the same curvature. Figure 7 Demonstrates the approximate binarization function Under the same curvature (k=10), compared It is closer to the standard binarized function.

[0073]

[0074] In step S502, the adaptive approximate binarization function is:

[0075] In this invention, an approximate binarization function that adaptively approximates the standard binarization function is designed.

[0076]

[0077] Here, δ is the similarity factor, δ∈(0,1). When δ is sufficiently small, the approximation function can approach the standard binarization function. This also means that a suitable δ can improve the detection accuracy of the model. Therefore, in order to adaptively determine δ, this invention incorporates δ as an optimizable variable into the training of the segmentation network. In this way, δ can be adaptively adjusted and the approximation function can evolve towards the standard binarization function during training.

[0078] In form, the adaptive training process can be treated as a loss minimization problem of the segmentation network, as shown in Equation (6), where x is the network input and y is the network output:

[0079]

[0080] Where λ represents the preset parameter.

[0081] Therefore, the gradient of δ during backpropagation can also be calculated and automatically adjusted using L2 regularization constraints.

[0082]

[0083] Therefore, in the differentiable binarization module, formula (5) can be transformed into formula (8) as the approximate binarization function of the present invention.

[0084]

[0085] in, Let P represent an approximate binary image, T represent a probability image, and i,j represent the horizontal and vertical coordinate values ​​of each of the three images, i.e., the pixel positions of the images.

[0086] In step S6, obtaining the text bounding box through the bounding box generation module specifically involves:

[0087] Text bounding boxes can be constructed by the bounding box generation module during the inference phase. In fact, probabilistic graphs and approximate binary graphs can generate almost identical text bounding boxes. Considering efficiency, probabilistic graphs are used to construct text bounding boxes. The formation of bounding boxes mainly involves three steps:

[0088] 1) Use a fixed threshold of 0.1 to convert the probabilistic map or approximate binary map into a binary map;

[0089] 2) Generate connected regions from the binary graph;

[0090] 3) The offset Δ obtained using the Vatti Clipping algorithm ′ To expand connectivity. Δ ′ The calculation formula is as follows:

[0091]

[0092] Among them, A ′ L is the area of ​​the reduced polygon region. ′ It is the perimeter of the reduced polygon region, r ′ It is a hyperparameter that can be adjusted according to different datasets, with an adjustment range of 1.5 to 2.5.

[0093] Example

[0094] To further verify the processing performance of the method of the present invention, further explanation is provided using relevant verification data.

[0095] First, the dataset is set as follows:

[0096] ICDAR 2015 is a publicly available text dataset containing many small, low-resolution text instances. It consists of 1500 natural scene images taken with Google Glass at a resolution of 720×1280, divided into 1000 training images and 500 test images. The text instances in the dataset are labeled at the word level.

[0097] Digital meter datasets are collected using Android mobile devices in power industry production environments (such as...). Figure 8 As shown, a total of 1400 images were generated. All images were uniformly scaled to 1080×1920 resolution, comprising 1000 training images and 400 test images. The NRSS method identified 1150 images as low-quality, including 820 in the training set and 330 in the test set. Considering practical needs, only certain designated display areas were labeled for text instances; the remaining text instances were treated as background.

[0098] ResNet-18_vd and ResNet-50_vd, pre-trained on the ImageNet 1K classification dataset, were used as backbone networks. All models were trained for 2000 epochs on both datasets with a batch size of 16. A cosine learning rate decay strategy was followed during training, with an initial learning rate of 0.001. The Adam optimizer was used for optimization, with a first exponential decay rate β1 of 0.9 and a second exponential decay rate β2 of 0.999.

[0099] During the training phase, the following data augmentation strategies were used to augment the training data: (1) random rotation within the range of [-10°, 10°]; (2) random scaling; and (3) random left and right flipping. To improve training efficiency, the size of all adjusted images was uniformly adjusted to 640×640.

[0100] During the inference phase, the input image size for the test set was uniformly adjusted to 736×1280. The inference batch size was set to 1, and a single thread using a 3090 GPU was used for testing. Inference time includes the time spent on forward computation of the model and the time spent on post-processing operations, with post-processing operations accounting for approximately 30% of the inference time.

[0101] In this embodiment, two no-reference evaluation methods, NRSS and NIQE, were used to conduct experiments on digital instrument image quality evaluation indicators.

[0102] Table 1: Image quality assessment of the digital instrument dataset using the NRSS method. The output NRSS Score is divided into 10 uniform intervals. Combining the NRSS Score and subjective human judgment, the image quality is categorized into three levels: high, medium, and low. Based on subjective human judgment, the image begins to become blurry when the NRSS Score is greater than 0.4. Reviewing the original image revealed the following causes of low image quality: out-of-focus, partial obstruction of the image, lens shake during shooting, and reflections and shadows on the instrument panel. These factors are the causes of the low-quality image.

[0103] Table 1

[0104]

[0105]

[0106] Table 2: Distribution intervals of NIQE scores obtained using another no-reference evaluation method, NIQE, on the digital instrument dataset. The maximum NIQE score is 20.98, and the minimum is 5.46.

[0107] Table 2

[0108]

[0109] By comparing the NRSS and NIQE methods, it can be found that the NIQE method has no boundary constraints on its output, resulting in limited discriminative power. In contrast, thanks to the boundedness of the NRSS output (the results are normalized), the NRSS method provides a clearer understanding of the image quality distribution within the dataset.

[0110] Experiments show that NRSS can effectively classify the image quality of digital instrument datasets, and the output results are consistent with the subjective judgment of the human eye. Therefore, the NRSS method can be used as the evaluation method for low-quality images of digital instruments in this paper.

[0111] Table 3

[0112]

[0113] Table 3 shows that adding a bottom-up path to the FPN structure of DB Net resulted in a 0.55% performance improvement in the F1-measure on ICDAR2015. Based on this path enhancement, a feature fusion factor was added between the four adjacent layers P5 to P2. And select appropriate values ​​for them (as shown in Table 3). After that, the F1-measure improved by 0.83%, and the recall improved by 2.69%. The method proposed in this invention adds a fusion path to the DB Net structure, increasing the network complexity, but without a significant decrease in inference speed (inference speed is 50.85 FPS, only 0.5 FPS lower than DB Net), fully achieving real-time inference. Using ResNet-18 as the backbone network, the effectiveness of the FPN module was verified on the ICDAR 2015 dataset. The three fusion factors are listed from left to right as follows: This embodiment sets up the experiment with... and Two constraints were imposed, and several combinations were manually searched under these constraints, resulting in three sets of fusion factor parameters for comparison.

[0114] Table 4

[0115]

[0116]

[0117] Experiments were conducted on ICDAR 2015 using different backbone networks to validate the effectiveness of the adaptive approximate binarization function. When using ResNet-18, with a curvature set to 50, the sigmoid approximation function was 0.15%, 0.05%, and 0.24% lower in F1-measure, precision, and recall, respectively, compared to the tanh approximation function. However, when using the same tanh approximation function, F1-measure, precision, and recall all increased with increasing curvature. This result confirms the points raised in the methods section regarding the approximate binarization function.

[0118] In experiments using ResNet-50 as the backbone network, the adaptive approximate binarization function of this invention outperforms DB Net's approximate binarization function by 1.85% in F1-measure, 1.29% in accuracy, and 2.31% in recall, and is also 1.63 FPS faster in inference.

[0119] Table 5

[0120]

[0121] The method proposed in this invention was compared with existing methods on ICDAR 2015. As shown in Table 5, the experimental results of this invention achieve the best results in F1-measure and recall compared to existing methods, exceeding the best existing method by 1.06% in F1-measure and 2.06% in recall.

[0122] Table 6

[0123]

[0124]

[0125] The detection results of the proposed method and existing methods were compared on a digital instrument dataset. Retina DB, based on a ResNet-50 backbone network, showed the best performance among all methods. Some visualization results are shown below. Figure 9 As shown in Table 6, the method of this invention achieves the highest levels in F1-measure, accuracy, and recall. When using ResNet-18 as the backbone network, our method also surpasses DBNet to achieve the fastest inference speed, reaching 47.13 FPS, which can meet the requirements of real-time detection of data instrument readings.

[0126] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

[0127] The above descriptions are merely some embodiments of the present invention. Those skilled in the art can make various modifications and improvements without departing from the inventive concept of the present invention, and these all fall within the scope of protection of the present invention.

Claims

1. A method for detecting digital instrument readings based on low-quality images, characterized in that, Includes the following steps: Step S1: Acquire the images captured by the digital instrument and perform image preprocessing to obtain the image dataset of the digital instrument; Step S2: Input the images in the image dataset into the feature extraction network for multi-scale feature extraction; Step S3: The feature extraction network extracts multi-scale feature maps, which are then input into the Feature Pyramid Network (FPN) module for feature fusion to obtain the feature map. ; Step S4, through feature map Predicting the generation probability map and threshold map ; Step S5, plot the probability graph and threshold map The approximate binary image is calculated by feeding it into the adaptive differentiable binarization module. ; Step S6: Obtain the text bounding box of the image acquired by the digital instrument through the bounding box generation module, and then generate the text bounding box from the approximate binary image. Extract the display area image; In step S2, the FPN module uses a bottom-up path enhancement structure to perform feature fusion on multi-scale feature maps. Define in a bottom-up direction The feature map of the i-th layer is represented in a top-down order, according to the formula. Obtain the first fusion result of the i-th layer, where, M represents the number of scales extracted by the feature extraction network, and the first fusion result is only calculated up to the second layer; Indicates the channel matching used. Convolution operation, This indicates a 2x upsampling operation used for size matching. Indicates from the first layer to the first Layer fusion factor; The second fusion result is generated in a bottom-up order. ,in, , will the Layer After downsampling, and compared with the first The first fusion result of the layer By adding element by element, we get the first... The second fusion result of the layers ; In step S5, the approximate binary image... Specifically: in, Represents the similarity factor. , , , Representing approximate binary graphs Probability graph Threshold map At pixel point ( ) value.

2. The method as described in claim 1, characterized in that, In step S2, a ResNet-18 or ResNet-50 backbone network is used as the feature extraction network.

3. The method as described in claim 1, characterized in that, In step S2, the feature extraction network extracts feature maps at four scales, with each feature map having a size of 1 / 2, 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the input image of the feature extraction network.

4. The method as described in claim 1, characterized in that, In step S6, the probability graph is... Or an approximate binary image The bounding box generation module obtains the text bounding box of the image acquired by the digital instrument.

Images