Text recognition and detection model method based on improved MobileNetV3 interface

Abstract

The invention relates to the technical field of text recognition and detection, in particular to a text recognition and detection model method based on an improved MobileNetV3 interface, and the method comprises the steps: collecting a data set, collecting a large number of pictures of an oil and gas industry software interface, and labeling the data set as a neural network training sample; constructing a text recognition and detection model, wherein a backbone network of the text recognition and detection model is a MobileNetV3 model of an NAM attention mechanism module embedded at the tail end of each network block; and training the model, and converting the trained text recognition detection model into a text reasoning model for storage. According to the method, a backbone network of a recognition detection model is improved, an attention mechanism is improved by NAM through contribution factors of weights, image features are highlighted by utilizing variance measurement of the weights of the training model, and a text detector, a direction classifier and a human recognizer are introduced, so that the detection speed and the detection accuracy of the model are improved; and the high coverage rate requirement of modern software automatic testing is met.

Description

Technical Field

[0001] The present invention relates to the technical field of text recognition and detection, and in particular to a method for text recognition and detection model based on an improved MobileNetV3 interface. Background Art

[0002] The intelligent oil and gas operation platform interface is an intelligent management and control system for the oil and gas industry. By integrating advanced technologies such as the Internet of Things, big data, cloud computing, and artificial intelligence, it can achieve comprehensive monitoring, real-time data analysis, intelligent decision support, and automated control of the oil and gas production process. The platform can significantly improve oil and gas production efficiency, reduce operating costs, optimize resource allocation, and improve production safety. Software automation testing, as an important automated testing method for the digitalization and intelligence of Industry 4.0, uses an automated testing tool to verify the needs of various software tests. It includes the management and implementation of testing activities, and the development and execution of test scripts. Intelligent software automatic testing is conducive to the unified establishment of industry-customized software development standards, greatly improving testing efficiency, ensuring software security, and improving software quality, empowering the digital transformation and intelligent development of the industrial Internet.

[0003] The software automation testing of the intelligent oil and gas operating platform interface participates in the entire process of data collection and transmission to value generation in the oil and gas industry chain. It is an essential carrier for the digital transformation of the oil and gas industry and a core component of the oil and gas industrial Internet. At present, domestic software testing is still dominated by manual testing, with low reliability and work efficiency and difficult to achieve full coverage. Intelligent software automatic testing is conducive to the unified establishment of industry customized software development standards, greatly improving testing efficiency, ensuring software security, and improving software quality, empowering the digital transformation and intelligent development of the oil and gas industry. Summary of the invention

[0004] The present invention aims to solve one of the problems existing in the background technology.

[0005] To this end, the present invention provides a method for interface text recognition and detection model based on an improved MobileNetV3.

[0006] The technical solution adopted by the present invention to solve its technical problem is:

[0007] A method for interface text recognition and detection model based on an improved MobileNetV3 includes the following steps:

[0008] S1, collect data sets, collect a large number of pictures of oil and gas industry software interfaces, and annotate the data sets as neural network training samples;

[0009] S2, constructing a text recognition detection model, wherein the backbone network of the text recognition detection model is a MobileNetV3 model with a NAM attention mechanism module embedded at the end of each network block;

[0010] S3, train the model, and convert the trained text recognition detection model into a text inference model for storage.

[0011] Furthermore, in step S1, data set annotation includes data cleaning, data preprocessing, data annotation, establishing annotation specifications, data review and correction, and data division.

[0012] Furthermore, in the process of cleaning the data, the image data that is irrelevant to the identified object and erroneous image data in the image data set are removed, and the data preprocessing includes binarization and grayscale processing of the image.

[0013] Furthermore, in the process of data labeling, the PPOCRLabel automatic labeling tool is used to label the text with a rectangular box and four points.

[0014] Furthermore, a text detector based on a simple segmentation network of DBNet is introduced into the text recognition detection model. The DBNet extracts features through the backbone network, and uses the Neck structure of DBFPN to fuse the features of each stage to obtain features. The fused features are decoded by operations such as convolution of the Head part to generate a probability map and a threshold map, and the two are fused and calculated to obtain an approximate binary map.

[0015] Furthermore, a direction classifier based on an improved PACT quantization algorithm is introduced into the text recognition detection model, and the activation functions of the activation value preprocessing of the improved PACT quantization method include a RELU function and a hardswish function.

[0016] Furthermore, a text recognizer based on a CRNN model is introduced into the text recognition detection model, and the CRNN model combines a CNN model and a RNN model.

[0017] Furthermore, in step S3, the trained text recognition detection model is converted into an inference model for storage.

[0018] A computer device comprising:

[0019] processor;

[0020] A memory, the memory being used to store executable instructions;

[0021] Wherein, the processor is used to read the executable instructions from the memory and execute the executable instructions to implement the improved MobileNetV3 interface text recognition detection model method as described in any one of claims 1 to 8.

[0022] A computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the processor implements the method for interface text recognition and detection based on an improved MobileNetV3 model as described in any one of claims 1 to 8.

[0023] The beneficial effects of the present invention are as follows: the present invention embeds a NAM attention mechanism at the end of each network block in MobileNetV3; uses DBNet as a text detector based on a simple segmentation network to obtain text area polygon information; uses an improved PACT quantization algorithm as a direction classifier; and uses a CRNN model to perform text recognition on a rectangular box containing a text area. The present invention improves the backbone network of the recognition and detection model by adding a NAM attention mechanism, introduces DBNet as a text detector, introduces PACT as a direction classifier, and introduces CRNN as a text recognizer, thereby improving the model detection speed and detection accuracy, and meeting the high coverage requirements of modern software automated testing. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0025] Figure 1 The present invention provides a flowchart of the method for constructing an industrial software text recognition model.

[0026] Figure 2 It is the DBNet network workflow diagram in the present invention.

[0027] Figure 3 This is the CRNN network workflow diagram in the present invention.

[0028] Figure 4 This is a text recognition effect diagram of the text recognition detection model in the present invention. DETAILED DESCRIPTION

[0029] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, which only illustrate the basic structure of the present invention in a schematic manner, and therefore only show the components related to the present invention.

[0030] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In addition, features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "multiple" means two or more.

[0031] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0032] A method for interface text recognition and detection model based on an improved MobileNetV3 includes the following steps: S1 collects a data set

[0033] A large number of pictures of oil and gas industry software interfaces are collected, and the text in them is annotated as a dataset for preprocessing and annotating as neural network training samples.

[0034] Manually take screenshots of the software interface of the oil and gas industry and manually annotate the text that needs to be recognized in the screenshots. Annotation usually includes operations such as adding labels, classifications, or annotations to the data so that the data can be correctly recognized and understood when training the model. The annotated data set is used to train, verify, and test the improved model.

[0035] S2 dataset annotation

[0036] Dataset annotation is the process of processing raw data to meet the needs of deep learning. Dataset annotation includes data cleaning, data preprocessing, data annotation, establishing annotation specifications, data review and correction, and dividing training, verification and test sets:

[0037] S21 Data Cleaning

[0038] Before labeling, the dataset must be cleaned to remove image data that is irrelevant to the object to be identified and erroneous image data in the image dataset to ensure data quality and accuracy and improve the recognition accuracy of the model.

[0039] S22 Data Preprocessing

[0040] After cleaning, the screenshots of the oil and gas industry software interface must be properly preprocessed, such as binarization and grayscale processing to ensure the training quality of the model.

[0041] S23 Data Annotation

[0042] According to the required recognition task, the text in the pre-processed screenshots of the oil and gas industry software interface is manually labeled or annotated to mark and classify the raw data so that it can be used for model training.

[0043] In order to ensure the consistency and accuracy of annotation, it is necessary to establish clear annotation specifications to annotate the text in the screenshots of the oil and gas industry software interface. In view of the characteristics of the oil and gas industry software interface, such as many types of text and different font sizes, the PPOCRLabel automatic annotation tool is used to annotate the text with rectangular frames and four points.

[0044] S24 Data Review and Correction

[0045] After the labeling is completed, the text labels in the screenshots of the oil and gas industry software interface need to be reviewed and corrected, and manual adjustments need to be made to the defective and missing labels in the automated labeling so that the labeling results can be used for detection and recognition model training.

[0046] S25 data division

[0047] Finally, the labeled screenshots of the oil and gas industry software interface are divided into training, validation and test sets according to the needs. The training set is used to train the model, the validation set is used to adjust the model parameters and select the best model, and the test set is used to evaluate the generalization ability of the model.

[0048] S3 builds text recognition detection model

[0049] In MobileNetV3, a NAM attention mechanism is embedded at the end of each network block. NAM stands for Normalization-based Attention Module. Similar to the CBAM attention mechanism, the NAM attention mechanism adopts the integration mode of channel attention + spatial attention, but the NAM attention mechanism redesigns the two attention modules so that a lower weight penalty factor can be used to reduce the weight of less significant features, making the overall attention weight more efficient while maintaining the same performance in calculation, alleviating the contradiction between model accuracy and model lightweight. For the MobileNetV3 of the present invention, a NAM attention mechanism is embedded at the end of each network block, and the importance of weights is expressed by using the scale factor of batch normalization and the standard deviation, thereby avoiding the addition of fully connected layers and convolutional layers in attention mechanisms such as SE and CBAM.

[0050] S31 introduces text detector

[0051] Use DBNet as a text detector based on a simple segmentation network to obtain text area polygon information. The workflow is as follows: Figure 2 As shown. DBNet extracts features through the backbone network, and uses the Neck structure of DBFPN (feature pyramid networks) to fuse the features of each stage to obtain features. The fused features are decoded by the convolution of the Head part and other operations to generate probability maps and threshold maps, and the two are fused to calculate an approximate binary map. The feature fusion part DBFPN of the Neck structure is similar to the FPN structure in the target detection task. It fuses feature maps of different scales to improve the detection effect of text areas of different scales. In order to facilitate the merging of feature maps of different channels, the present invention uses 1×1 convolution to reduce the feature maps to the same number of channels.

[0052] S32 introduces direction classifier

[0053] The improved PACT quantization algorithm is used as a direction classifier to classify the direction of the text instance detected by the text, and the text is rotated to 0 degrees before being sent to the subsequent text recognizer. PACT quantization is a new quantization method that minimizes the accuracy loss caused by model quantization by removing some outliers before quantizing the activation value, so that the model can learn more appropriate quantization parameters. The activation value preprocessing of the ordinary PACT quantization method is based on the RELU function, and its formula is: In the formula, the part of x whose value is greater than 0 is truncated, and the truncation threshold is α.

[0054] All activation values ​​greater than a certain threshold are reset to a constant. However, the MobileNetV3 network activation function in the present invention also includes hardswish. Using ordinary PACT quantization will result in higher precision loss. In order to reduce the quantization loss, the activation function formula is modified as follows: In the formula, the parts of x that are greater than 0 and less than 0 are truncated, and the truncation thresholds are α and -α respectively.

[0055] The improved PACT quantization algorithm not only truncates the distribution greater than 0, but also imposes the same restrictions on the part less than 0, so as to better obtain the range to be quantized and reduce the quantization loss. At the same time, the truncation threshold is a trainable parameter. The model will automatically find a reasonable truncation threshold during the quantization training process to further reduce the quantization accuracy loss.

[0056] S33 introduces text recognizer

[0057] The CRNN model is used to perform text recognition on the rectangular box containing the text area, and the text alignment problem is solved by processing indefinite length sequences in an end-to-end manner. The CRNN model combines the CNN model and the RNN model. CNN is used to extract image features, and RNN processes the features extracted by CNN to obtain output, corresponding to the final label. CRNN contains three layers, namely convolution layer, recurrent layer and transcription layer. Although the length of English words in each image is inconsistent, the length of the features extracted after CNN is fixed, so only one transcription layer is needed to obtain the final result. The workflow is as follows: Figure 3 As shown in the figure: first, the image is preprocessed, where the image height must be a multiple of 16; secondly, CNN is used to extract image convolution features; then the extracted image convolution features are sent to LSTM to extract sequence features; finally, CTC is introduced to solve the problem of character alignment during training, so that training can be performed without a one-to-one correspondence between labels and outputs.

[0058] S4 converts the trained text recognition and detection model into a text inference model for storage, so that it can be directly called for recognition and detection. The inference model is generally a solidified model that saves the model structure and model parameters in a file during model training, and is mostly used in prediction and deployment scenarios; the model saved during the training process is a checkpoints model, which only saves the model parameters and is mostly used for training recovery, etc. Compared with the checkpoints model, the inference model will additionally save the model's structural information, has superior performance in prediction deployment and accelerated reasoning, and is flexible and convenient, so it is more suitable for actual system integration.

[0059] The following table shows the text recognition detection model using the existing MobileNetV3 model and the text recognition detection model in this application. Figure 4 (a) shows a comparison table of recognition results of software interface recognition in the oil and gas industry. The table uses mAP (average accuracy of all categories) to evaluate the accuracy of text recognition of the two models. Compared with the existing MobileNetV3 model, the MobileNetV3 model that introduces the NAM attention mechanism as the backbone network of the text recognition model has a significantly higher accuracy in text recognition detection than the existing MobileNetV3 model. The text recognition detection results of the text recognition detection model in this article are shown in Figure 2. Figure 4 As shown in panel b.

[0060] Model <![CDATA[mAP 0.5:0.95 ]]> <![CDATA[mAP 0.5 ]]> <![CDATA[mAP 0.75 ]]> <![CDATA[mAR max =100]]> Original 0.730 0.954 0.868 0.768 Modified 0.805 0.978 0.929 0.874

[0061] In summary, the present invention improves the backbone network of the recognition detection model, respectively uses DBNet as a text detector, uses an improved PACT quantization algorithm as a direction classifier, and uses a CRNN model for text recognition, thereby improving the model detection speed and detection accuracy, and meeting the high coverage requirements of modern software automated testing. In the present invention, by adding the NAM attention mechanism to MobileNetV3 as the backbone network of the text recognition model, NAM improves the attention mechanism through the contribution factor of the weight, and uses the variance measurement of the training model weight to highlight the image features; DB is used as a text detector based on a simple segmentation network, and the binary threshold is added to the training to learn so as to fuse multi-scale features, detect multi-scale text and obtain a faster detection speed; the improved PACT quantization algorithm is used as a direction classifier, and the activation value range is continuously clipped to reduce the outliers in the activation distribution, so that the quantization model can obtain a more reasonable quantization range to reduce the quantization loss; CRNN is used to recognize the detected text, and the text alignment problem is solved by processing indefinite length sequences in an end-to-end manner.

[0062] Based on the above ideal embodiments of the present invention, the relevant staff can make various changes and modifications without departing from the technical concept of the present invention through the above description. The technical scope of the present invention is not limited to the contents of the specification, but must be determined according to the scope of the claims.

Claims

1. A method for interface text recognition and detection model based on an improved MobileNetV3, characterized in that: The following steps are included: S1, collect data sets, collect a large number of pictures of oil and gas industry software interfaces, and annotate the data sets as neural network training samples; S2, constructing a text recognition detection model, wherein the backbone network of the text recognition detection model is a MobileNetV3 model with a NAM attention mechanism module embedded at the end of each network block; S3, train the model, and convert the trained text recognition detection model into a text inference model for storage.

2. The method for interface text recognition and detection model based on the improved MobileNetV3 according to claim 1 is characterized in that: In step S1, data set annotation includes data cleaning, data preprocessing, data annotation, establishing annotation specifications, data review and correction, and data division.

3. The method for interface text recognition and detection model based on the improved MobileNetV3 according to claim 2 is characterized in that: In the process of cleaning data, the image data that is irrelevant to the identified object and the erroneous image data in the image data set are removed. Data preprocessing includes binarization and grayscale processing of the image.

4. The method for interface text recognition and detection model based on the improved MobileNetV3 according to claim 2 is characterized in that: In the process of data labeling, the PPOCRLabel automatic labeling tool is used to label the text with rectangular boxes and four points.

5. The method for interface text recognition and detection model based on the improved MobileNetV3 according to claim 1 is characterized in that: The text recognition detection model introduces a text detector based on a simple segmentation network of DBNet. The DBNet extracts features through a backbone network, and uses the Neck structure of DBFPN to fuse the features of each stage to obtain features. The fused features are decoded by operations such as convolution of the Head part to generate a probability map and a threshold map, and the two are fused and calculated to obtain an approximate binary map.

6. The method for interface text recognition and detection model based on the improved MobileNetV3 according to claim 2 is characterized in that: The text recognition detection model introduces a direction classifier based on an improved PACT quantization algorithm, and the activation functions of the activation value preprocessing of the improved PACT quantization method include a RELU function and a hardswish function.

7. The method for interface text recognition and detection model based on the improved MobileNetV3 according to claim 2 is characterized in that: The text recognition detection model introduces a text recognizer based on a CRNN model, and the CRNN model combines a CNN model and a RNN model.

8. The method for interface text recognition and detection model based on the improved MobileNetV3 according to claim 1 is characterized in that: In step S3, the trained text recognition detection model is converted into an inference model for storage.

9. A computer device, characterized in that: include: processor; A memory, the memory being used to store executable instructions; Wherein, the processor is used to read the executable instructions from the memory and execute the executable instructions to implement the improved MobileNetV3 interface text recognition detection model method as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the processor implements the method based on the improved MobileNetV3 interface text recognition detection model as described in any one of claims 1 to 8.

Images