Industrial instrument type identification and reading method

Abstract

The invention relates to the field of instrument reading, and particularly discloses an industrial instrument type identification and reading-oriented method, which comprises the following steps of S1, detecting the position of an instrument dial plate through an improved target detection network, cutting according to the position of an instrument to obtain a position image of the instrument, and completing classification; s2, through a template matching algorithm, pointer rotation center information and measuring range information of the pointer type instrument are obtained through detection from the position image; through the target detection network YOLOv5 and improved modules (such as Mosai c data enhancement, a C3ECA module, FPN and PAN structures and the like) thereof, the instrument detection speed and accuracy are remarkably improved, the calculation amount is reduced, and the instrument can be quickly and accurately positioned in a complex environment; for a pointer type instrument, through a template matching algorithm and Hough transform straight line detection, in combination with an angle method and piecewise linearization calculation, the position of a pointer can be accurately identified, and the instrument reading can be calculated.

Description

Technical Field

[0001] The invention belongs to the field of instrument reading, in particular to a method for industrial instrument type identification and reading. Background Art

[0002] In the field of industrial production, instruments play a vital role. They can accurately measure key parameters such as temperature, pressure, flow and liquid level to ensure that the equipment operates in the best condition. By continuously monitoring various parameter indicators, the instrument can keenly capture abnormal conditions of the equipment and trigger the early warning mechanism, so that the enterprise can respond quickly, adjust the production process, effectively prevent the occurrence of production accidents, and ensure production safety and continuity. In addition, the instrument also has powerful data recording and storage functions. They record and save the data generated in the production process, providing a valuable database for subsequent data analysis. These historical data are not only an important basis for enterprises to evaluate production efficiency, but also a key factor in promoting the optimization of production processes and improving product quality. By analyzing these data, staff can gain insight into subtle changes in the production process and formulate more scientific and reasonable production strategies, thereby achieving a dual improvement in production efficiency and product quality.

[0003] In industrial environments, instruments are mainly divided into two categories: pointer instruments and digital instruments, each of which has unique advantages and applicable scenarios. Pointer instruments are known for their simple structure, low cost and excellent anti-interference ability. They are suitable for situations that require real-time monitoring and intuitive display, such as equipment operation status monitoring and control systems. The instant feedback capability of pointer instruments allows operators to quickly grasp key information and make corresponding adjustments. In contrast, digital instruments are characterized by high accuracy, low reading error, and convenience of data recording and transmission. They can provide more accurate and detailed measurement results, meeting the needs of scenarios with high requirements for measurement accuracy. In complex environments such as automated production lines, the data processing capabilities of digital instruments are particularly critical. They can easily realize data recording, analysis and remote transmission, providing strong support for the intelligent and refined management of the production process. In addition, the display interface of digital instruments is clear and easy to read, which reduces human errors and improves work efficiency and accuracy.

[0004] Due to the complex industrial environment, some meters have to rely on manual reading due to the lack of communication interfaces. This traditional method is not only costly and inefficient, but also easily affected by human factors, resulting in frequent reading errors. Harsh environmental conditions such as high temperature and high radiation further increase the difficulty and risk of manual meter reading. Summary of the invention

[0005] To solve the above technical problems, the present invention provides a method for industrial instrument type recognition and reading, so as to solve the problems in the prior art that the traditional method not only has high cost and low efficiency, but is also extremely vulnerable to human factors, resulting in frequent reading errors.

[0006] A method for industrial instrument type recognition and reading includes the following steps:

[0007] S1. Detect the position of the instrument dial through an improved object detection network, crop the position image of the instrument according to the instrument position, and complete the classification;

[0008] S2. Detect the pointer rotation center information and range information of the pointer-type instrument from the position image through a template matching algorithm;

[0009] S3. Detect the pointer information of the pointer-type instrument from the position image through Hough transform line detection;

[0010] S4. Identify the instrument reading of the pointer-type instrument through the pointer rotation center information of the pointer-type instrument, the range information, and the pointer information of the pointer-type instrument;

[0011] S5. Detect the character position of the digital instrument from the position image through OCR character detection for extraction and reading.

[0012] Preferably, step S1 specifically includes:

[0013] S11. Use a number of instrument image sample data sets to train the object detection network model to obtain a trained neural network model;

[0014] S12. Obtain the instrument picture to be detected of the instrument;

[0015] S13. Input the instrument picture to be detected into the object detection network;

[0016] S14. Use the Mosaic data augmentation module of the object detection network to perform a series of data augmentation methods such as random scaling, random cropping, and random arrangement on the instrument image to obtain an instrument feature enhanced image;

[0017] S15. Use the adaptive picture scaling module of the object detection network to uniformly scale the input instrument image to obtain a group of input images with the same size;

[0018] S16. Use the Focus structure and CSP structure of the target detection network Backbone module to perform slicing and convolution operations on the instrument image, enabling the model to learn more features and obtain three feature maps of different sizes. Incorporate the ECA module into the C3 module to form a new C3ECA module, and embed the C3ECA module into the hierarchy before the feature pyramid pooling layer in the backbone network, making the detector more focused on the dial and effectively suppressing noise.

[0019] S17. Use the FPN structure and PAN structure of the target detection network Neck module to perform a bottom-up operation to convey strong localization features on the instrument image, aggregate features from different backbone layers to different detection layers, and enhance the feature fusion ability of the network.

[0020] S18. Use the Prediction module of the target detection network, adopt the improved loss function EIOU-Loss as the loss function for the bounding box, and evaluate the error between the predicted value and the true value. Adopt non-maximum suppression to discard the prediction boxes with lower scores, and obtain the instrument dial position image and the instrument type.

[0021] S19. Use the coordinates of all instrument dials obtained by the target detection network, and use the cropping method to crop the detected dial images. Crop an image with multiple instrument dials into multiple individual instrument images for subsequent reading recognition.

[0022] Preferably, step S2 specifically includes:

[0023] Input the coordinates of the rotation center of the template pointer and the range information of the set template from the position image.

[0024] Preferably, step S3 specifically includes:

[0025] Use the K-means binarization algorithm to obtain the binarized image of the pointer-type instrument from the position image.

[0026] Use the line detection algorithm to detect the longest line in the pointer-type instrument image from the position image, which is the position corresponding to the pointer.

[0027] Preferably, step S4 specifically includes:

[0028] Use the angle method to calculate and output the actual angle of the pointer-type instrument from the position image.

[0029] Use piecewise linearization and template matching to calculate and output the actual angle of the non-linear pointer-type instrument.

[0030] Preferably, step S5 specifically includes:

[0031] Using the CRNN text recognition model, calculate the actual reading of the digital instrument from the position image and output it.

[0032] Compared with the prior art, the present invention has the following beneficial effects:

[0033] Through the object detection network YOLOv5 and its improved modules (such as Mosaic data augmentation, C3ECA module, FPN and PAN structures, etc.), the detection rate and accuracy of the instrument are significantly improved, the amount of calculation is reduced, and the instrument can be quickly and accurately located in a complex environment;

[0034] For analog instruments, through the template matching algorithm and Hough transform line detection, combined with the angle method and piecewise linearization calculation, the pointer position can be accurately identified and the instrument reading can be calculated;

[0035] For digital instruments, through the OCR character detection technology, characters can be quickly and accurately recognized and the readings can be extracted;

[0036] The present invention is applicable to the detection and reading recognition of various types of instruments, including analog instruments and digital instruments, and can handle images under different lighting, angles and noise conditions, with strong adaptability and broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 is the basic process of analog instrument and digital instrument recognition and reading;

[0038] Figure 2 is the structural diagram of the object detection network YOLOv5;

[0039] Figure 3 is the structural diagram of the ECA attention mechanism module;

[0040] Figure 4 is to identify the position of the instrument dial and the type of instrument category through the object detection network;

[0041] Figure 5 is the basic flow chart of analog instrument recognition and reading;

[0042] Figure 6 is the basic flow chart of digital instrument recognition and reading;

[0043] Figure 7 is the recognition flow chart of non-linear instruments;

[0044] Figure 8 is to input the instrument picture into the object detection network and obtain the display of the recognition result of the instrument classification. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0045] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0046] As Figures 1 to 8 shown:

[0047] The technical solution of the present invention is as follows:

[0048] Detect the position of the instrument through a target detection network, crop the position image of the instrument according to the instrument position and complete classification;

[0049] Detect the pointer rotation center information and range information of the pointer-type instrument from the position image through a template matching algorithm.

[0050] Detect the pointer information of the pointer-type instrument from the position image through Hough line detection;

[0051] Identify the instrument reading of the pointer-type instrument through the pointer rotation center information and range information of the pointer-type instrument, and the pointer information of the pointer-type instrument;

[0052] Detect the character position of the digital instrument from the position image through OCR character detection for extraction and reading.

[0053] In the above detection method of the pointer-type instrument, the steps of detecting the position of the instrument through a target detection network, cropping the position image of the instrument according to the instrument position and completing classification include:

[0054] Use a number of instrument image sample datasets to train the target detection network model to obtain a trained neural network model;

[0055] Obtain the instrument picture to be detected of the instrument;

[0056] Input the instrument picture to be detected into the target detection network YOLOv5;

[0057] Use the Mosaic data augmentation module of the target detection network to perform data augmentation on the instrument image in a way of random scaling, random cropping and random arrangement to obtain an instrument feature enhanced image; Adaptive image scaling reduces the filling of black edges at both ends of the image, reduces the computational amount, and improves the target detection rate;

[0058] Using the CSP structure of the target detection network Backbone module, the ECA module is fused into the C3 module to form a new C3ECA module. The C3ECA module is embedded into the level before the feature pyramid pooling layer in the YOLOv5 backbone network, making the detector more focused on the dial and effectively suppressing noise;

[0059] Using the Focus structure and CSP structure of the target detection network Backbone module, slicing operation and convolution operation are performed on the instrument image to obtain feature maps with different sizes; Two CSP structures are designed in YOLOv5s. The CSP1_X structure is used in the Backbone main network, and the CSP2_X structure is used in the Neck; The original 608×608×3 image becomes a 304×304×32 feature map after slicing operation and a convolution operation with a 32 convolutional kernel.

[0060] Using the FPN structure and PAN structure of the target detection network Neck module, an operation of conveying strong localization features from bottom to top is performed on the instrument image, and feature aggregation is performed on different detection layers from different backbone layers to enhance the feature fusion ability of the network; A feature pyramid containing two PAN structures is added after the FPN.

[0061] Using the target detection network Prediction module, the improved loss function EIOU-Loss is used as the loss function of the bounding box to evaluate the error between the predicted value and the true value. The optimized bounding box loss function has a faster convergence speed than the original one, which can improve the regression accuracy and thus enhance the model detection accuracy; Non-maximum suppression is adopted to discard the prediction boxes with lower scores to obtain the position image and type of the instrument.

[0062] In the above detection method of the pointer-type instrument, through the template matching algorithm, the pointer rotation center information and range information of the pointer-type instrument are detected from the position image, including:

[0063] Using the template matching algorithm, input the coordinates of the template pointer rotation center and the range information of the set template into the position image; Template matching is to search for the target in a large image. The template is the known target to be found in the image, and the target has the same size and direction as the template.

[0064] In the above detection method of the pointer-type instrument, through the Hough transform line detection, the pointer information of the pointer-type instrument is detected from the position image, including:

[0065] Using the K-means binarization algorithm, a binarized image of the pointer-type instrument is detected from the position image; K-Means binarization is an image processing method based on the clustering algorithm K-Means, which converts a color image into a binary image. In K-Means binarization, each pixel in the image is regarded as a data point, and then the K-Means algorithm is used to divide these pixels into two clusters, namely black and white;

[0066] Using the Hough transform line detection, the longest line in the pointer-type instrument image is detected from the position image, which is the position corresponding to the pointer; The Hough transform is an algorithm that is inevitably encountered in image processing. It detects objects with a specific shape through a voting algorithm. In this process, a set that conforms to the specific shape is obtained as the Hough transform result by calculating the local maximum of the cumulative result in a parameter space;

[0067] In the above detection method of the pointer-type instrument, the instrument reading of the pointer-type instrument is identified through the pointer rotation center information, the range information, and the pointer information of the pointer-type instrument, including:

[0068] Using the angle method, the actual angle of the pointer-type instrument is calculated and output from the position image. Its specific value is calculated through the pointer position, range, and rotation angle.

[0069] Using piecewise linearization, with the upper left corner of the instrument image as the origin, the horizontal direction as the x-axis, and the vertical direction as the y-axis, a rectangular coordinate system is established. According to the deflection angle value θ of the line where the pointer is located detected by the line detection relative to the horizontal negative direction, judge the deflection angle of the corresponding adjacent main scale line relative to the horizontal negative direction Furthermore, the corresponding adjacent main scale value α is obtained i+1 、α i . Substitute these variables into the formula to obtain the final reading of the pointer instrument. Among them, α i+1 、α i represent the values of the left and right adjacent scale lines of the pointer respectively, represent the deflection angle values of the lines connecting the left and right adjacent scale lines to the rotation center relative to the horizontal negative direction respectively, θ represents the deflection angle value of the pointer relative to the horizontal negative direction, and α is the final real number. By solving the above formula, the reading of the pointer instrument can be obtained.

[0070] In the above detection method of the digital instrument, through the character recognition method, the character positions of the digital instrument are detected from the position image for extraction and reading, including:

[0071] Use a camera to obtain an image with text, and then preprocess the obtained image to improve the recognition accuracy and identify the text area in the image;

[0072] Use an object detection algorithm to determine the position of the characters.

[0073] As Figure 1 and Figure 2 shown:

[0074] Embodiment 1

[0075] The present invention discloses a neural network for identifying instrument types, based on the YOLO network. The YOLOv5s network includes an input layer Input, a backbone network Backbone, a neck network Neck, and a head network Prediction; the Input layer uses Mosaic data augmentation of random scaling, random cropping, and random arrangement, which can well adapt to small target detection, enriches the data set, and reduces the computing power requirements of the training GPU; adaptively calculates the optimal anchor box values in different training sets; adopts adaptive image scaling, reduces the padding of black edges at both ends of the image, reduces the amount of calculation, and improves the target detection rate. The Backbone main network uses the CSP1_X structure, and the Neck uses the CSP2_X structure; the ECA module is fused into the C3 module to form a new C3ECA module, and the C3ECA module is embedded into the level before the feature pyramid pooling layer in the YOLOv5 backbone network. This design can make the detector more focused on the dial and effectively suppress noise; adopts the FPN+PAN structure, FPN conveys strong semantic features from top to bottom, and a feature pyramid containing two PAN structures is added behind FPN to convey strong localization features from bottom to top, and aggregates features from different backbone layers to different detection layers to strengthen the feature fusion ability of the network; Prediction: adopts EIOU_Loss as the loss function of the bounding box to evaluate the error between the predicted value and the true value, and solves the problem that the boundaries do not overlap; adopts non-maximum suppression (NMS) to discard the predicted boxes with lower scores. Yolov5 needs the center point coordinates and width and height of the target to crop the detected target, so it is necessary to save the dial information; in the txt file for saving target information, from left to right, it represents: target type, x value of the target center point coordinates, y value, width and height of the target. The conditions for cropping are to obtain x1, x2, y1, y2. Since the coordinates stored in the txt file of yolov5 are all normalized, the calculated x1, x2, y1, y2 values are all values after normalization and need to be restored to their original values according to a ratio.

[0076] As Figure 4 and Figure 5 shown:

[0077] Example 2

[0078] Through the template matching algorithm, the pointer rotation center information and range information of the pointer-type instrument are detected from the position image; the type of the instrument to be measured is determined through the target detection network. For example, it is determined as a pressure gauge, an ammeter or a voltmeter, and then its pointer rotation center and range information are set according to the set instrument template. In order to identify a variety of different instruments, all templates of the instruments to be measured need to be selected as references in the present invention.

[0079] As Figure 3 shown:

[0080] Example 3

[0081] Through the Hough transform line detection, the pointer information of the pointer-type instrument is detected from the position image; then the pointer line is obtained through skeleton thinning and least squares linear fitting. The Hough transform adopts a transformation method between two coordinate spaces, mapping curves or lines with the same shape in one space to a point in another coordinate space, thereby forming a peak, and transforming the problem of detecting any shape into the problem of statistical peak; according to the deflection angle value of the line where the pointer is located detected by the line detection relative to the horizontal negative direction, the deflection angle of the corresponding adjacent main scale line relative to the horizontal negative direction is judged, and then the corresponding adjacent main scale value is obtained, and the final pointer instrument reading is obtained by substituting it into the angle method formula.

[0082] As Figure 6 and Figure 7 shown:

[0083] Example 4

[0084] Through the pointer rotation center information and range information of the pointer-type instrument, and the pointer information of the pointer-type instrument, the instrument reading of the pointer-type instrument is identified, and the actual angle of the pointer-type instrument is calculated and output from the position image using the angle method. Calculate its specific value through the pointer position, range and rotation angle; through piecewise linearization, taking the upper left corner of the instrument image as the origin, the horizontal direction as the x-axis, and the vertical direction as the y-axis, a rectangular coordinate system is established. According to the deflection angle value θ of the line where the pointer is located detected by the line detection relative to the horizontal negative direction, judge the deflection angle of the corresponding adjacent main scale line relative to the horizontal negative direction and then obtain the corresponding adjacent main scale value α i+1 、α i , substitute these variables into the formula Obtain the final pointer instrument indication. By solving the above equation, the reading of the pointer instrument can be obtained. In the above detection method of the digital instrument, through the OCR character recognition method, the character position of the digital instrument is detected from the position image for extraction and reading, including obtaining an image with text through a camera, and then preprocessing the obtained image to improve the recognition accuracy and identify the text area in the image; using an object detection algorithm to determine the position of the character; using a CRNN character recognition model to output the digital information of the recognized character.

[0085] All standard parts used in the present invention can be purchased from the market. The special-shaped parts can be customized according to the description in the specification and the drawings. The specific connection methods of each part all adopt conventional means such as bolts, rivets, and welding that are mature in the prior art. The machinery, parts, and equipment all adopt conventional models in the prior art. Coupled with the circuit connection adopting the conventional connection method in the prior art, details are not described herein. The content not described in detail in this specification belongs to the prior art well-known to those skilled in the art.

[0086] In the description of the present invention, the terms "first" and "second" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. The meaning of "plurality" is two or more, unless otherwise specifically defined.

[0087] In the present invention, unless otherwise clearly specified and defined, the terms "mounted", "connected", "connected to", "fixed" and other terms should be understood in a broad sense. For example, it may be a fixed connection, a detachable connection, or integrated; it may be a mechanical connection or an electrical connection; it may be directly connected or indirectly connected through an intermediate medium, and it may be the internal communication of two components or the interaction relationship between two components. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0088] In the present invention, unless otherwise clearly specified and defined, the first feature being "on" or "under" the second feature may be that the first and second features are in direct contact, or the first and second features are indirectly in contact through an intermediate medium. Moreover, the first feature being "above", "over" and "on top of" the second feature may be that the first feature is directly above or obliquely above the second feature, or merely indicates that the first feature has a higher horizontal height than the second feature. The first feature being "under", "beneath" and "underneath" the second feature may be that the first feature is directly below or obliquely below the second feature, or merely indicates that the first feature has a lower horizontal height than the second feature.

[0089] In the description of this specification, the description referring to terms such as "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0090] In the drawings of the disclosed embodiments of the present invention, only the structures related to the disclosed embodiments are involved, and other structures can refer to the general design. Without conflict, the same embodiment and different embodiments of the present invention can be combined with each other.

[0091] Although the present invention has been described in detail with reference to the foregoing embodiments, for those skilled in the art, they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for industrial instrument type identification and reading, characterized in that: The following steps are involved: S1. Detect the position of the instrument dial through the improved target detection network, crop the instrument position image according to the instrument position and complete the classification; S2. Detecting the rotation center information and range information of the pointer type instrument from the position image through a template matching algorithm; S3, detecting the pointer information of the pointer instrument from the position image through Hough transform straight line detection; S4, identifying and obtaining the instrument reading of the pointer instrument through the rotation center information of the pointer instrument, the range information, and the pointer information of the pointer instrument; S5. Through OCR character detection, the character position of the digital instrument is detected from the position image for extraction and reading.

2. A method for industrial instrument type identification and reading as claimed in claim 1, characterized in that: Step S1 specifically includes: S11, using a number of instrument image sample data sets to train the target detection network model to obtain a trained neural network model; S12, obtaining a picture of the instrument to be tested; S13, inputting the image of the instrument to be detected into the target detection network; S14, using the Mosaic data enhancement module of the target detection network, a series of data enhancement methods such as random scaling, random cropping, and random arrangement are applied to the instrument image to obtain an instrument feature enhanced image; S15, using the adaptive image scaling module of the target detection network to uniformly scale the input instrument images to obtain a group of input images of the same size; S16. Use the Focus structure and CSP structure of the Backbone module of the target detection network to perform slicing and convolution operations on the instrument image, so that the model can learn more features and obtain feature maps of three different sizes; fuse the ECA module into the C3 module to form a new C3ECA module, and embed the C3ECA module into the layer before the feature pyramid pooling layer in the backbone network, so that the detector can focus more on the dial and effectively suppress noise; S17, using the FPN structure and PAN structure of the target detection network Neck module, perform bottom-up transmission of strong positioning feature operations on the instrument image, perform feature aggregation on different detection layers from different backbone layers, and enhance the feature fusion capability of the network; S18. Use the Prediction module of the target detection network and adopt the improved loss function EIOU-Loss as the loss function of the bounding box to evaluate the error between the predicted value and the true value; use non-maximum suppression to discard the prediction box with a lower score, and obtain the instrument dial position image and instrument type; S19. Using the coordinates of all the instrument panels obtained by the target detection network, the detected instrument panel images are cropped using a cropping method, and an image with multiple instrument panels is cropped into multiple separate instrument images to facilitate subsequent reading recognition.

3. A method for industrial instrument type identification and reading as claimed in claim 1, characterized in that: Step S2 specifically includes: Input the coordinates of the template pointer rotation center and set the template range information from the position image.

4. A method for industrial instrument type identification and reading as claimed in claim 1, characterized in that: Step S3 specifically includes: Using K-means binarization algorithm, a binarized image of the pointer instrument is obtained from the position image; Using the straight line detection algorithm, the longest straight line of the pointer instrument image is detected from the position image, which is the position corresponding to the pointer.

5. A method for industrial instrument type identification and reading as claimed in claim 1, characterized in that: Step S4 specifically includes: Using the angle method, the actual angle of the pointer instrument is calculated from the position image and output; Using piecewise linearization and template matching, the actual angle of the nonlinear pointer instrument is calculated and output.

6. A method for industrial instrument type identification and reading as claimed in claim 1, characterized in that: Step S5 specifically includes: The CRNN character recognition model is used to calculate the actual reading of the digital instrument from the position image and output it.

Images