Water meter reading recognition method based on deep learning and application of water meter reading recognition method in water meter verification

Abstract

The invention discloses a water meter reading recognition method based on deep learning and application of the water meter reading recognition method in water meter verification. The water meter reading recognition method comprises the steps that a dial plate is positioned through a dial plate positioning image processing algorithm based on feature extraction and parameter space; a deep learning neural network model is built and training is carried out to complete identification of a magnitude pointer; the method comprises the following steps: firstly, constructing a data set containing multiple types of magnitude pointer images; performing data preprocessing on the image; further processing the image by adopting a data enhancement technology; a neural network model is built, the architecture of the model comprises a convolution layer, a pooling layer, a discarding layer, a flattening layer and a full connection layer, an input image is subjected to convolution, pooling and discarding operation after normalization, features are extracted and compressed from an original image, and output is completed through the full connection layer; and finally identifying the pointing angle of the water meter plum-blossom needle. According to the invention, the reading of the water meter is automatically identified, the complexity of manual operation is reduced, and the data accuracy and the verification reliability can be improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of water meters, and in particular relates to a water meter reading recognition method based on deep learning and its application in water meter calibration. Background Art

[0002] As a measuring instrument managed by the state in accordance with the law, water meters must undergo strict metrological verification before being put on the market. At present, the verification method of manual reading still dominates, and small-caliber water meters usually use the start-stop volume method. In this method, the verification personnel need to read the readings of the water meter and the standard container at the start and end time of each verification point, and calculate the indication error. This process is not only time-consuming, but also requires the verification personnel to perform a large amount of data recording and calculation. Long-term work can easily lead to visual fatigue and human errors, thereby affecting the efficiency and accuracy of the verification. Therefore, under the premise of ensuring accuracy, improving the efficiency of water meter verification is of great significance to its market launch.

[0003] In the process of water meter calibration, the traditional manual reading method is widely used, but it is inefficient and prone to errors. In order to improve the efficiency of calibration and reduce human errors, image processing technology can be used to automatically identify the readings of the water meter level pointer and plum blossom needle. By automatically capturing and analyzing the water meter level pointer image, it can not only reduce the tediousness of manual operation, but also improve the accuracy of data and the reliability of calibration. The application of this technology helps to speed up the market launch of water meters and complete the calibration work efficiently and with high quality.

[0004] In order to solve the problem of water meter calibration efficiency, many companies have gradually tried to use electronic technology for automatic calibration. For example, the method of using photoelectric sensors for automatic calibration is to obtain the water meter flow reading by measuring the pulse signal generated by the plum blossom needle during the rotation process. The water meter and the photoelectric sensor correspond one to one. When the water flows through the water meter, the plum blossom needle rotates. The photoelectric sensor completes the counting of the plum blossom needle pulse signal, thereby realizing the automatic calibration of the water meter. This method improves the automation and efficiency of water meter calibration. However, its anti-interference ability to vibration and light changes still needs to be improved, and it is easy to lose pulse signals. These problems affect the actual application effect of this method.

[0005] Therefore, the prior art lacks a water meter reading recognition method based on deep learning for water meter calibration, and a deep learning neural network is used to realize automatic recognition of water meter pointer readings. Summary of the invention

[0006] In view of the above-mentioned deficiencies in the prior art, the technical problem to be solved by the present invention is: how to provide a water meter reading recognition method based on deep learning, apply deep learning neural network to realize automatic recognition of water meter pointer readings, reduce the tediousness of manual operation, and improve data accuracy and verification reliability.

[0007] In order to solve the above technical problems, the present invention adopts the following technical solution: a water meter reading recognition method based on deep learning, including pointer area reading recognition, and the pointer area reading recognition includes the following steps:

[0008] 1. Dial positioning

[0009] An image processing algorithm for dial positioning based on feature extraction and parameter space. The algorithm first performs a global analysis on the input original dial image through color feature extraction to preliminarily obtain regional information containing dial data, which may contain noise. Then, the morphological-based image processing technology is used to perform regional fusion and noise removal on the preliminarily obtained data area to obtain an area containing only dial data. Finally, by transforming the parameter space of the dial data area, the dial data position can be accurately located, thereby improving the positioning accuracy of the dial.

[0010] 1.1. Image acquisition and preprocessing: First, obtain the original image of the dial and perform preprocessing, including grayscale conversion and noise filtering of the image, to lay the foundation for subsequent feature extraction and image analysis.

[0011] 1.2. Color feature extraction: By analyzing the color distribution of the dial image, the color feature extraction technology is used to globally analyze the original image to preliminarily determine the possible area where the dial data is located. In this step, the HSV color space can be used for more accurate color analysis to distinguish the dial data area from the non-dial data area.

[0012] 1.3. Morphological processing: For the preliminary dial data area obtained by color feature extraction, the morphological image processing algorithm is applied to perform regional fusion and noise removal. This step removes small noise points in the image through operations such as expansion and corrosion, while enhancing the continuity of the dial data area.

[0013] 1.4. Parameter space transformation and dial data positioning: Map the processed dial data area to the parameter space, and use the Hough circle transform method to detect and locate the dial data position. In the parameter space, each point represents the center and radius of a possible circular dial data area. The position and size of the dial data area are accurately determined through accumulator statistics and threshold judgment.

[0014] 1.5. Result verification and optimization: Finally, the positioning results are verified and optimized. According to the actual application requirements, the algorithm parameters are adjusted, such as the threshold of the Canny edge detector and the accumulator resolution of the Hough transform, to adapt to various image conditions and improve the accuracy of positioning.

[0015] Through the above operation, the dial data can be extracted from the complex dial background, and the required data area can be obtained for further processing. Here, the magnitude pointer area and plum blossom needle area of ​​the water meter are extracted for the next step of identification.

[0016] 2. Reading recognition

[0017] The magnitude pointer area and plum blossom needle area of ​​the water meter extracted above are used to complete the recognition of the water meter magnitude pointer reading and the water meter plum blossom needle pointing angle.

[0018] 2.1 Identification of water meter level pointer reading

[0019] There are two main types of magnitude pointer recognition: recognition based on traditional image processing technology and recognition based on deep learning.

[0020] Although traditional image processing techniques can work effectively under certain conditions, their limitations when processing complex images are mainly manifested in the following aspects: First, these techniques are highly dependent on environmental conditions, such as lighting, background complexity, etc. Once these conditions change, the performance of the algorithm may drop significantly. Second, traditional methods require manual adjustment of multiple parameters, such as thresholds, filter sizes, etc., which is not only time-consuming but also inefficient, especially when dealing with diverse application scenarios. In addition, due to their reliance on predefined rules and parameters, these techniques are difficult to adapt to new scenarios that are different from the training environment, limiting the breadth and flexibility of their application. Finally, some complex image processing algorithms are computationally intensive and may not meet the performance requirements for real-time or near-real-time application scenarios.

[0021] In contrast, deep learning technology can automatically learn complex features from large amounts of data by building deep neural networks, without manually setting rules or adjusting parameters. This learning ability enables deep learning models to demonstrate excellent adaptability and accuracy when faced with diverse and unknown images. Deep learning models can also continuously optimize and improve their performance through continuous learning, especially in areas such as image recognition, object detection, and visual tracking, which have become a hot topic for research and application. Therefore, when highly accurate and reliable image processing solutions are required, choosing deep learning can not only improve efficiency, but also significantly improve the quality of results. Therefore, the identification of pointers is completed by building a deep learning model and training it.

[0022] In order to train the deep learning model, we first built a dataset containing various types of pointer images. These images cover different lighting conditions and background complexities to ensure that the model can perform well in various environments. Data preprocessing includes image cropping, grayscale conversion, and normalization to adapt to the model input requirements.

[0023] In order to further improve the generalization and robustness of the model, a variety of data enhancement techniques are used. Image rotation is a commonly used enhancement method. By rotating the image at different angles, the scene of the pointer in different positions is simulated to enhance the model's ability to recognize the change of the pointer direction. Image scaling can help the model adapt

[0024] Pointers and dashboards of different sizes. In addition, random noise and blur effects are added to simulate image quality problems that may be encountered in actual use, such as camera shaking and inaccurate focus. These enhancement techniques not only improve the model's ability to adapt to new environments, but also enhance its robustness in the face of various challenges that may be encountered in actual applications.

[0025] The neural network model is built. After the input image is normalized, it undergoes multiple convolution, pooling, and drop operations to effectively extract and compress features from the original image, and finally completes the output through the fully connected layer. The architecture of the model includes multiple convolutional layers, pooling layers, dropout layers, a flattening layer, and a fully connected layer. The input layer accepts a 32×32 pixel color image. The batch normalization layer normalizes the input image, which helps the stability and accelerated convergence during the model training process. Convolutional layer (Conv2D): The convolution kernel is used to extract image features. The output dimension of this layer is kept at 32×32 and the depth is 32. The convolutional layer can effectively capture local features such as edges, corners, etc. Max pooling layer (MaxPooling2D): This layer reduces the dimensionality of the 32×32 feature map to 8×8, and the depth is still 32. The pooling layer helps reduce the amount of computation and the risk of overfitting by reducing the dimensionality of the feature space while retaining important features. Dropout layer is added after each pooling layer to randomly discard the activation values ​​of some neurons with a certain probability, which helps to prevent overfitting of the model and enhance the generalization ability of the model. Flatten layer flattens the multi-dimensional output into one dimension so that it can be used as the input of the fully connected layer. This operation converts the 8×8×32 output into a one-dimensional vector of 2048 units.

[0026] The final fully connected layer (Dense) is used to output the final result.

[0027] 2.2 Identification of the water meter plum blossom needle pointing angle

[0028] Firstly, the plum blossom needle image is preprocessed using adaptive threshold segmentation technology. This technology is different from the traditional global threshold segmentation. It can dynamically adjust the threshold according to the illumination and texture characteristics of each small area of ​​the image, thereby effectively coping with the challenge of uneven illumination and ensuring the effective separation of the plum blossom needle from the background.

[0029] The segmented image is further subjected to probabilistic Hough line transform for line detection. Probabilistic Hough line transform is an optimized version of Hough transform. It reduces the consideration of all points in the image through random sampling, significantly improves the processing speed, and can effectively detect lines in the presence of complex backgrounds or more noise. During the detection process, a specific algorithm is designed to exclude falsely detected lines and only retain the line information that matches the plum blossom needle feature.

[0030] Finally, the accurate position and direction of the plum blossom needle are determined by analyzing the retained straight line information. Assuming that the top of the image is due north, the angle difference between the plum blossom needle and the due north direction is calculated to accurately obtain the plum blossom needle's pointing angle. By combining the adaptive threshold segmentation and probabilistic Hough line transform method, efficient and accurate plum blossom needle positioning is achieved, applying the powerful capabilities of traditional image processing technology in specific application scenarios to reduce the computational burden.

[0031] 3 Application in water meter calibration

[0032] Since the pointer of the water meter does not fit the disk completely, when reading at a certain height, the projection of the pointer on the disk does not always point completely to the actual value due to the viewing angle, and the reading of the last magnitude pointer often needs to be estimated, and there will be a small error in the estimation, so the recognition result of the magnitude pointer often deviates. Since the plum blossom needle itself carries numerical information and can be accurately located, the data of the plum blossom needle is introduced to solve the problem of errors caused by the inability to accurately estimate the minimum magnitude, and the plum blossom needle is used in the process of water meter calibration.

[0033] The result obtained by subtracting the readings that may have errors is converted into the result obtained by calculating the number of turns and angles of the plum blossom needle. The water meter calibration is completed by taking an image of the water meter dial at the beginning and end, and comparing the difference between the two magnitude pointer readings and the actual flow through the water meter during this period. If only the water meter pointer data is used, the two readings of the last pointer may have inevitable estimates, resulting in error accumulation, causing a large deviation in the result. By introducing the plum blossom needle data, the difference between the two positions of the plum blossom needle is used instead of the difference between the two estimated readings. Due to the characteristics of the plum blossom needle with clear background, distinct structure and accurate positioning, the accuracy of the difference between the two readings can be improved.

[0034] Significant effect: The present invention provides a water meter reading recognition method based on deep learning, which uses a deep learning neural network to realize automatic recognition of water meter pointer readings, reduces the tediousness of manual operation, and improves data accuracy and verification reliability. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 The water meter dial image;

[0036] Figure 2 Determine the image of the circle for the Hough circle transform;

[0037] Figure 3 Obtaining pointer region images for connected domain analysis;

[0038] Figure 4 The result image of the image segmentation area;

[0039] Figure 5 The pointer segmentation image before connected domain analysis;

[0040] Figure 6 It is the pointer segmentation image after connected domain analysis;

[0041] Figure 7 These are multiple ring images obtained by Hough circle transform;

[0042] Figure 8 It is the Hough circle image processed by precise positioning algorithm;

[0043] Fig. 9 Schematic diagram of neural network;

[0044] Fig.10 The model training results;

[0045] Fig.11 This is the recognition effect diagram of the model;

[0046] Fig.12 Schematic diagram of finding the location of plum blossom needles, where (a) is a schematic diagram of the plum blossom needle location result; (b) is the plum blossom needle image before adaptive threshold segmentation, (c) is the image after adaptive threshold segmentation; (d) is the plum blossom needle image detected by probabilistic Hough line transform, (e) is the plum blossom needle image after merging and clustering, and (f) is the plum blossom needle image after removing the wrong straight line;

[0047] Fig.13 The pictures of the water meter dial before and after calibration are as follows;

[0048] Fig.14 This is a schematic diagram of water meter calibration error;

[0049] Fig.15 The dial image is tilted at an angle;

[0050] Fig.16 This is a schematic diagram of feature point selection based on ORB image positioning algorithm;

[0051] Fig.17 It is the image aligned by the ORB-based image positioning algorithm;

[0052] Fig.18 The image is the final cut of the ROI fixed on the image;

[0053] Fig.19 It is the pixel distribution map of the water meter reading area after grayscale;

[0054] Fig. 20 It is a binary digital image after fixed threshold segmentation;

[0055] Fig.21 It is a binary digital image after adaptive threshold segmentation;

[0056] Fig. 22 The binary digital image is obtained by combining adaptive threshold segmentation and fixed threshold segmentation;

[0057] Fig.23 This is a digital image of a gear after segmentation using adaptive thresholding;

[0058] Fig.24 Digital images extracted for conversion to color space;

[0059] Fig.25 The training results of the character wheel area reading recognition model;

[0060] Fig.26 The figure is a flow chart of the method of the present invention. DETAILED DESCRIPTION

[0061] The present invention will be further described in detail below in conjunction with the accompanying drawings.

[0062] 1 Introduction

[0063] As a measuring instrument managed by the state in accordance with the law, water meters must undergo strict metrological verification before being put on the market. The current manual verification method consumes a lot of manpower and time costs. Automatic recognition of water meter readings is the basis of automated water meter verification. The present invention is based on image processing technology and image recognition technology, uses Hough circle transform and HSV color space to complete image segmentation, uses deep learning neural network to complete pointer recognition, uses probabilistic Hough line transform to complete plum blossom needle recognition, and finally introduces the data of the plum blossom needle to correct the pointer reading error, thereby realizing automatic recognition of water meter readings. It can be applied to automatic verification of water meters.

[0064] The present invention proposes an identification method for water meters with complex dials with multiple pointers. Image processing technology is used to accurately identify the water meter pointers and plum blossom needle readings. The reading information of the flow rate flowing through the water meter can be obtained by using the pointer image recognition results twice, thereby achieving high-precision calibration of the water meter and improving the calibration efficiency of the water meter.

[0065] 2 Principle and technical route

[0066] Mechanical water meters usually have multiple pointers, and the pointer shapes are relatively complex. Figure 1 The water meter image shown is composed of four magnitude pointers, and the indicated magnitudes from left to right are ×0.0001, ×0.001, ×0.01, and ×0.1, respectively, in cubic meters. The water meter reading can be read by the indication of the magnitude pointer, and the accuracy of the reading depends on the resolution of the minimum reading.

[0067] like Figure 2-Figure 26 As shown, the present invention discloses a water meter reading recognition method based on deep learning and its application in water meter calibration, which is used for water meter reading recognition. First, the pointer area reading is recognized. The pointer area reading recognition includes the following steps:

[0068] Step A: The dial is located using a dial location image processing algorithm based on feature extraction and parameter space. First, the input original dial image is globally analyzed by color feature extraction to preliminarily obtain regional information of the dial. Then, the morphological-based image processing technology is used to perform regional fusion and noise removal on the preliminarily acquired data area to obtain an area containing only the dial data. Finally, the dial data position, i.e., the magnitude pointer area and plum blossom needle area of ​​the water meter, is accurately located by transforming the parameter space of the dial data area.

[0069] Step B: Complete the recognition of the magnitude pointer reading and the pointing angle of the plum blossom needle of the water meter;

[0070] Step B1: The magnitude pointer is identified by building a deep learning neural network model and training it; first, a data set containing two or more types of magnitude pointer images is constructed; the images are preprocessed to adapt to the model input requirements; data enhancement technology is used to further improve the generalization ability and robustness of the model; a neural network model is built, and the architecture of the model includes a convolution layer, a pooling layer, a drop layer, a flattening layer and a fully connected layer. The input image is normalized and then convolved, pooled and dropped, features are extracted and compressed from the original image, and finally the output is completed through the fully connected layer;

[0071] Step B2: Identify the pointing angle of the plum blossom needle of the water meter;

[0072] Firstly, the plum blossom needle image is preprocessed using adaptive threshold segmentation technology, and the threshold is dynamically adjusted according to the illumination and texture characteristics of each small area of ​​the image to separate the plum blossom needle from the background.

[0073] The segmented image is further subjected to line detection through probabilistic Hough line transform. During the detection process, a corresponding algorithm is designed to exclude the wrongly detected lines and only retain the line information that matches the plum blossom needle feature.

[0074] Finally, the exact position and direction of the plum blossom needle are determined by analyzing the retained straight line information.

[0075] Image processing technology and image recognition technology play an important role in the identification of pointer readings. Image processing mainly involves image transformation, image enhancement and restoration, image segmentation and image classification, while image recognition includes template matching and neural network-based applications.

[0076] This paper studies image processing technology and image recognition technology, applies deep learning neural network to realize automatic recognition of pointer instrument readings, and improves the accuracy of readings through the recognition of plum blossom pointers, avoiding complex video stream processing. The main tasks are as follows:

[0077] 1) Design an image segmentation method to segment the area related to the pointer;

[0078] 2) Reading and identifying the extracted area;

[0079] 3) Conduct application tests on the results of the algorithm.

[0080] 3 Water meter pointer and plum blossom needle recognition based on deep learning

[0081] 3.1 Image Segmentation

[0082] Image segmentation plays an extremely important role in the field of image processing. Its main purpose is to extract meaningful targets from images. This technology is widely used in various types of image processing and is the basis of target recognition. Image segmentation technology mainly includes threshold segmentation, edge detection, region growing and matching detection technology. In the present invention, image segmentation technology is used to extract the internal sub-dial from the entire image. According to the characteristics of the water meter, a method combining Hough circle transform and color segmentation is designed, and the pointer area is located by connected domain analysis. By mutual calibration of the two positioning results, the precise positioning of the pointer area is completed.

[0083] The basic idea of ​​Hough circle transform is to transform the circular features in the image space into parameter space for expression. In the parameter space, each point represents the parameters of a circle that may exist in the image space (including the center and radius). After edge detection of the image, each edge point can correspond to a series of circles in the parameter space. Through the accumulator method, the number of times each parameter combination is repeatedly recognized can be counted, and finally the circle existing in the image can be determined through threshold judgment.

[0084] The Hough circle transform is used for circle detection, and its effect is affected by multiple parameters, including accumulator resolution, minimum circle center distance, high and low thresholds of the Canny edge detector, and circle radius range. Different parameter settings will lead to different detection results. In addition, environmental factors such as light may also affect the detection effect, such as Figure 2 As shown, multiple rings may be identified. Therefore, further methods are required to accurately locate the ring during use.

[0085] When processing images of color pointer water meters, the color segmentation method can effectively extract the magnitude pointer image from the complex background. In this process, choosing a suitable color space is crucial to the segmentation effect and efficiency. Compared with the RGB model, the HSV color space is more suitable for such tasks. The RGB model expresses color through a combination of light intensities of red, green, and blue colors, which does not intuitively reflect hue, saturation, and brightness. In contrast, the HSV color space decomposes color information into hue, saturation, and brightness, making color adjustment and processing more intuitive and convenient. Using the HSV color space for color recognition and segmentation can process images more accurately. After the connected domain analysis is performed on the area obtained by color segmentation, the desired pointer area can be retained, thereby achieving more accurate image processing results. The final obtained magnitude pointer area is as follows Figure 3 As shown; Figure 3 Obtain magnitude pointer region images for connected domain analysis.

[0086] Design a corresponding precise positioning algorithm to process the data obtained from the above two steps. The two sets of data are verified against each other to obtain the final segmentation area. The segmentation effect is shown in Figure 4 As shown, Figure 4 is the result image of the image segmentation area.

[0087] Preferably, the most intuitive method in theory is to use Hough circle transform to detect circles. In practice, it is found that the above situation will occur, that is, Figure 2 As shown, multiple rings may be identified. The expected effect cannot be achieved, so further methods are needed. The further method is color segmentation, which extracts the red color. The connected domain analysis is a further processing of the extracted red color, which merges the adjacent red colors into the same area, and then obtains the centroid of the area to represent the area.

[0088] The purpose of connected domain analysis is to obtain a more stable effect, even if some red is extracted due to uncontrollable factors (such as Figure 5 After analysis, we get Figure 6 It can also represent the centroid of this area. Figure 5 , Figure 6 It is only used to illustrate the purpose of connected domain analysis. Figure 5 Segment the image for magnitude pointers before connected domain analysis; Figure 6 The image is segmented by the magnitude pointer after connected domain analysis.

[0089] Preferably, a corresponding precise positioning algorithm is designed to process the data obtained from the above two steps, including:

[0090] 1. The circles detected by Hough circle transform whose centers are close to the center of mass are retained;

[0091] 2. Get a new point by averaging the positions of the centers of these circles and the centroids, calculate the average radius of this point and the circles from this point to the remaining points, and remove circles with radii smaller than the average radius;

[0092] 3. Calculate the average position of the circle center and the center of mass again to get a new point. Repeat step 2 above multiple times to get the final circle. Calculate the average position of the circle center and the center of mass to get the final center. Calculate the average radius of the final center to get the final circle.

[0093] Data comparison and verification is from Figure 7 Optimized Figure 8 ; Figure 7 These are multiple ring images obtained by Hough circle transform; Figure 8 This is the Hough circle image processed by precise positioning algorithm.

[0094] 3.2 Magnitude Pointer Identification

[0095] Before deep learning was widely used in image recognition, there were four methods for automatic reading of common pointer instruments: step length method, circular grayscale detection method, Hough transform method, and multi-feature matching method based on grayscale information, which mainly relied on traditional image processing technology. These methods rely on predefined rules and parameters, and are often limited by their inherent algorithmic limitations and show low adaptability and accuracy on complex images. In contrast, deep learning neural networks can automatically extract and learn deep features of images by learning a large amount of image data. This ability to automatically learn features from data enables deep learning neural network models to significantly improve recognition accuracy and robustness when dealing with complex image recognition tasks.

[0096] In the face of the limitations of traditional methods, deep learning provides an alternative. This invention uses deep learning to identify magnitude pointers, and completes the identification of magnitude pointers by building and training a corresponding neural network model. The neural network model diagram is shown in Fig. 9 As shown in the figure, the model accepts a color image of 32 by 32 pixels, which is normalized and then enters the convolution layer to extract data features. The pooling layer reduces the feature space dimension while retaining important features. In order to prevent overfitting, a dropout layer is added after each pooling layer. After multiple convolutional pooling and dropout operations, the output of the model is completed through the flattening layer and the fully connected layer.

[0097] Build the above neural network model and train it. The training results are as follows Fig.10 As shown, Fig.10 The model training results are shown in Figure 2. The model loss on the test set of 1400 images is 0.0026, and the accuracy is 98.43%. The recognition effect of the model is shown in Figure 2. Fig.11 shown.

[0098] 3.3 Plum Blossom Needle Positioning

[0099] For the plum blossom needle positioning part, since the plum blossom needle has obvious structural features and the processing requirements are relatively simple, traditional image processing technology can provide a more efficient and intuitive solution. These technologies are excellent in extracting image features of clear structures and have low computational cost, so they are used for plum blossom needle positioning in the present invention.

[0100] Adaptive threshold segmentation is an image processing technique that, unlike global threshold segmentation, does not use a single threshold to process the entire image, but dynamically adjusts the threshold based on the characteristics of each region in the image. In adaptive threshold segmentation, the algorithm considers local features of the image, such as illumination changes and texture differences, to calculate a most suitable threshold for each small region. This method can more accurately distinguish different regions, especially in the case of uneven illumination, and can effectively improve the segmentation effect.

[0101] Probabilistic Hough Line Transform is an effective method for detecting straight lines in image analysis. This technique is an improved version of the traditional Hough Transform, which reduces the amount of calculation by random sampling, thereby increasing the processing speed. In the probabilistic Hough Line Transform, instead of considering all points in the image, points are randomly selected from a set of candidate points including edge points for line detection. This method not only reduces the consumption of computing resources, but also effectively detects straight lines in images with complex backgrounds or more noise.

[0102] In order to complete the positioning of the plum blossom needle, the plum blossom needle area obtained by the above image segmentation is first segmented by adaptive threshold to divide the image into foreground and background, and the plum blossom needle is extracted from the background. According to the characteristics of the plum blossom needle, the probabilistic Hough line transform is used to detect the straight lines in the pattern of the extracted plum blossom needle. The corresponding algorithm is designed to remove the wrongly detected straight lines. The final positioning result recognition result is as follows Fig.12 As shown in (a), assuming that the north direction is the initial position, the value of the plum blossom needle pointing angle can be calculated. The plum blossom needle reading can be determined by the plum blossom needle pointing angle. The plum blossom needle reading increases the reading by an order of magnitude. When only the magnitude pointer is used, the magnitude of 0.0001 needs to be estimated, and there is a reading error of about ±0.00005. Introducing the plum blossom needle position change angle as compensation can make the reading error less than ±0.000005.

[0103] Preferably, a corresponding algorithm is designed to remove the wrongly detected straight lines, including:

[0104] Using probabilistic Hough line transform to detect straight lines in the pattern, the result will be multiple straight lines, among which, Fig.12 (b) is the plum blossom needle image before adaptive threshold segmentation. Fig.12 (c) in the figure is the image after adaptive threshold segmentation; Fig.12 As shown in (d) in the figure, we can see Fig.12 (d) in the figure contains a large number of straight line segments, and an algorithm is required to remove unnecessary straight lines. First, merge the straight lines, merge adjacent straight lines together, and then calculate the distance of all straight lines (here the distance calculation uses the Euclidean distance formula). All similar straight lines form different clusters, and then calculate the central straight line of each cluster to obtain the central straight line of each cluster. This straight line represents the position of this cluster. The result is as follows: Fig.12 As shown in (e) in the figure, we can then search for two straight lines whose intersection points are close to the center of the image and are nearly vertical and retain them. The result is as follows Fig.12 As shown in (f) in .

[0105] 4 Applications

[0106] The dial reading information can be obtained by performing magnitude pointer recognition and plum blossom needle positioning after image segmentation, such as Fig.13 Shown are pictures of the water meter dial before and after calibration. Through the recognition of the magnitude pointer image, it can be concluded that the water meter reading is 0.00266 cubic meters. However, when reading the ×0.0001 magnitude pointer, due to the camera angle and the recognition of the magnitude pointer position, the maximum possible reading error is about ±0.00005. Therefore, the position change angle of the plum blossom needle is introduced in the calculation as compensation, which can make the reading error less than ±0.000005, thereby greatly improving the accuracy of the reading. Fig.13Two recognition processes are specifically demonstrated.

[0107] After completing the above algorithm and connecting to the water meter calibration platform for testing, the initial reading of the water meter is 126.82L. It is tested with 50L of water volume for 10 consecutive times, and the difference obtained by only the magnitude pointer indication and the difference calculated after introducing the plum blossom needle data are recorded respectively. Then, the difference calculated after introducing the plum blossom needle data and the 50L water volume are compared to obtain the relative error of the water meter. The test results are as follows: Fig.14 As shown, Fig.14 Schematic diagram of water meter calibration error.

[0108] The test results show that accurate readings were obtained by using the algorithm to identify the pointer readings 10 times in a row, and the standard deviation of the error was 0.15, which proves that the algorithm has good stability. By introducing the reading of the plum blossom needle into the water meter magnitude pointer reading, the reading accuracy of image recognition can be improved tenfold, further improving the minimum graduation value of water meter calibration.

[0109] Water meter wheel area reading recognition

[0110] The present invention uses a convolutional neural network for the task of recognizing water meter readings, and trains a corresponding neural network model for the recognition of the readings of the wheel area. First, data preparation is performed, which includes positioning and cropping the image. In the process of positioning the wheel area of ​​the water meter, the wheel area in the complex dial image needs to be segmented out. However, when the dial image is actually collected, the camera may be tilted at a random angle when installed, causing the collected image to also show a random angle of tilt, such as Fig.15 As shown. This will have a great impact on the positioning of the character wheel. In addition, in the process of character wheel area positioning, some interference factors need to be eliminated to obtain the precise area contained in the outer frame of the character wheel.

[0111] Image registration is an important technology in the field of computer vision. It aims to align and match multiple images to eliminate the spatial and geometric differences between them and align them in the same coordinate system. The methods of image registration are mainly divided into feature-based and region-based methods. The feature-based method extracts feature points or feature descriptors from the image and achieves image alignment by matching these feature points or feature descriptors. The registration method using feature matching can automatically find the correspondence between two images, avoiding the tedious process of manually selecting key points. Common features include corners, edges, and textures. The region-based method divides the image into different regions and achieves image alignment by matching these regions. Common region matching methods include mutual information and normalized cross-correlation. Through image registration, image alignment and matching can be achieved, image quality and accuracy can be improved, and a reliable basis for subsequent analysis and processing can be provided.

[0112] Based on the advantages and disadvantages of the above methods, this patent uses image registration in image positioning tasks and proposes an image positioning algorithm based on ORB (Oriented FAST and Rotated BRIEF). ORB is a feature point detection and descriptor generation algorithm based on FAST (Features from Accelerated Segment Test) and BRIEF (Binary Robust Independent Elementary Features) algorithms. It has the characteristics of fast calculation speed, suitable for real-time applications, and good scale invariance and rotation invariance.

[0113] By detecting the feature points in the image and calculating the descriptors of these feature points, the target image and the standard image are aligned based on these descriptors. The specific algorithm flow is as follows:

[0114] 1. Use ORB feature detector and descriptor generator to detect feature points in the two images and calculate descriptors;

[0115] 2. Use Hamming distance to match and obtain the matching point pairs between the two images;

[0116] 3. Filter out excellent matching point pairs;

[0117] 4. Find the geometric transformation relationship between the two images based on the matching point pairs;

[0118] 5. Use the RANSAC algorithm to calculate the homography matrix between the two images from the matching point pairs;

[0119] 6. Use the homography matrix to perform perspective transformation on the image to be registered to achieve image alignment;

[0120] After the image alignment is completed, the image is basically aligned with the standard image. In order to improve the accuracy of the alignment, the image is further aligned. The image is recognized again, the position of the predefined reference structure in the image is found, and the coordinates of the position point with the highest matching degree are returned. Template matching is used, and finally an affine transformation is performed based on the obtained position points to obtain a further calibration image.

[0121] like Fig.16 and Fig.17 As shown in the figure, for tilted and rotated images, the algorithm can still select better feature points and perform alignment transformation.

[0122] After completing the above alignment operation on the image, the fixed ROI of the image can be finally cut to obtain the image data required for subsequent tasks. The cutting result is as follows Fig.18 shown.

[0123] 2.2 Data Augmentation

[0124] In deep learning, the performance and generalization ability of the model are often limited by the quality and quantity of the training data. If the training data is too little or too single, the model may have the problem of overfitting, that is, it performs well on the training data but performs poorly on new data. To solve this problem, data augmentation can be used to expand the training data set. Data augmentation generates more training samples by performing a series of transformations and expansions on the original data. These transformations can simulate various changes and noises in the real world, so that the model can better adapt to different scenarios and conditions. By increasing the diversity of data, data augmentation can help the model learn more features and patterns, and improve the generalization ability and robustness of the model.

[0125] The present invention combines and transforms four data enhancement methods, namely geometric transformation, color transformation, random rotation and random scaling, to obtain a data enhanced image. The enhanced image data is omitted here.

[0126] 4-character wheel area reading recognition

[0127] Character recognition is one of the key parts of the dial reading system. Due to the interference of the bezel gear in the character wheel area, the image of the character wheel area has a lot of noise, and there are half characters in the numbers on the water meter wheel. Therefore, the image of the character wheel area cannot be directly used for the training of deep learning neural networks. Therefore, further research and processing of the image of the character wheel area is needed.

[0128] The pixels of the grayscale water meter are counted, such as Fig.19 As shown, it can be seen that the pixel distribution in the grayscale water meter reading area is more concentrated.

[0129] In order to separate the digital area from the background, if a fixed threshold is used directly, the concentrated pixel values ​​will cause the segmentation effect of the digital area and the background to be affected by the preset threshold. As the environment changes, different images will require different thresholds. Improper threshold setting will result in the following problems: Fig. 20 In the case shown, the image has shadows, which causes the numbers and the border to stick together after binarization, causing adverse effects on subsequent digital character recognition. It may cause the reading area to be greatly affected by the border or it may be impossible to distinguish between the numbers and the background. The local threshold binarization will be discussed below to find a suitable binarization scheme.

[0130] The local threshold binarization method is an improved binarization method that determines the threshold by considering the pixel values ​​around each pixel instead of using a global threshold. Using this method when processing complex images can improve the quality of the binarized image. Commonly used local threshold binarization methods include Otsu's Binarization, Local Adaptive Thresholding, Sauvola Binarization, etc.

[0131] In view of the above problems of using fixed threshold to segment binary images, combined with the characteristics of local threshold binarization method, this paper uses adaptive threshold binarization method, which divides the image into multiple local regions and then applies adaptive threshold binarization to each region. The threshold of each local region is calculated based on the local statistical information of the region. Fig.21 As shown in the figure, the adaptive threshold segmentation can effectively segment the numbers. However, the binary image of the adaptive threshold is affected by the border of the water meter reading area, and there is still a lot of noise interference around the numbers. This image cannot be used for deep learning neural network model training.

[0132] By combining adaptive threshold segmentation and fixed threshold segmentation, using adaptive threshold and dynamic adjustment of fixed threshold segmentation value, two threshold segmentation operations are performed on the same digital image, and the two images are overlapped to retain the common value to obtain an overlapping image, and the obtained overlapping image is expanded, such as Fig. 22 As shown in Figure 2, the processed image can reduce a lot of noise interference while completely retaining the digital image structure. The image after the dilation operation is as follows: Fig. 22 shown.

[0133] After the above processing, the digital image contains only a small amount of noise and can be used for the training of deep learning neural networks. However, for the water meter that needs to be identified, the extraction of the numbers in the reading area is not only affected by the border of the digital area, but also in the decimal part of the reading area. Due to the presence of visible gear characters, the gears are extracted as part of the numbers when the image is binarized using an adaptive threshold. Fig.23 As shown in the figure, this image cannot be used for deep learning neural network model training. Therefore, a separate processing process needs to be designed for the decimal part of the water meter.

[0134] Since the fractional part of the water meter is a color image, the significant feature of the color image is the increase in data volume and the increase in information carried compared to the grayscale image. Color image processing is closely related to the color space used. The appropriate color space can make full use of the color information of the image.

[0135] For the decimal part of the water meter, the present invention adopts a method of converting the color space to convert the digital image of the decimal part into the HSV color space and extract its red part, such as Fig.24 As shown in Figure 2, the digital extraction of the image after color space conversion has a strong ability to resist gear and edge interference.

[0136] VGG16 is a deep convolutional neural network model developed by the research team of the Visual Geometry Group of the University of Oxford and proposed in 2014. It consists of 16 convolutional layers and 3 fully connected layers, hence the name VGG16. Its convolutional layer consists of 13 consecutive convolutional layers, each of which uses a 3×3 convolution kernel and a ReLU activation function. This small-sized convolution kernel can provide more nonlinear expression capabilities, and the depth of the network can be gradually increased by stacking multiple layers. The pooling layer of VGG16 uses a 2×2 maximum pooling operation to reduce the spatial dimension of the feature map. After the convolutional layer and the pooling layer, VGG16 uses 3 fully connected layers, of which the last fully connected layer outputs the classification result of the network.

[0137] Since there are half characters in the digital readings of water meters, the present invention uses VGG16 to extract features from binary images in the neural network model for character wheel area reading recognition based on its simple and consistent network structure, good performance, and good image feature extraction capabilities. The local features of the numbers are learned to solve the problem of half-character recognition. Each binary image is converted into a feature matrix composed of eigenvalues. The feature matrix and the binary image correspond one to one. A fully connected neural network is then established, and the corresponding binary image feature matrix is ​​used to train the fully connected neural network to obtain a deep learning model with recognition capabilities. The training results of the model are shown in Figure 1. Fig.25 As shown in the figure, the reading recognition of the character wheel area is completed through the deep learning neural network model.

[0138] like Fig.25 As shown, the method designed by this patent uses VGG16 to extract features to construct a feature matrix and uses a deep learning neural network to learn the feature matrix. It has good results and only requires a small amount of training to achieve a high accuracy. The data set used by this model contains 1500 images, of which 1200 images are used as training sets and 300 images are used as test sets. After 10 trainings, the loss on the training set is 0.0311, the accuracy is 99.52%, and the loss on the test set is 0.0544, the accuracy is 98.54%.

[0139] The above are only preferred implementations of the present invention. It should be pointed out that a number of modifications and improved technical solutions made by those skilled in the art without departing from the technical solution should also be deemed to fall within the scope of protection required by the claims.

Claims

1. A water meter reading recognition method based on deep learning, characterized in that: Including pointer area reading recognition, the pointer area reading recognition includes the following steps: Step A: The dial is located using a dial positioning image processing algorithm based on feature extraction and parameter space. First, the input original dial image is globally analyzed by color feature extraction to preliminarily obtain the regional information containing the dial data. Then, the morphological-based image processing technology is used to perform regional fusion and noise removal on the preliminarily obtained data area to obtain an area containing only the dial data. Finally, the dial data position, i.e., the magnitude pointer area and plum blossom needle area of ​​the water meter, is accurately located by transforming the parameter space of the dial data area. Step B: Complete the recognition of the magnitude pointer reading and the pointing angle of the plum blossom needle of the water meter; Step B1: The magnitude pointer is identified by building a deep learning neural network model and training it; first, a data set containing two or more types of magnitude pointer images is constructed; the images are preprocessed to adapt to the model input requirements; data enhancement technology is used to further improve the generalization ability and robustness of the model; a neural network model is built, and the architecture of the model includes a convolution layer, a pooling layer, a drop layer, a flattening layer and a fully connected layer. The input image is normalized and then convolved, pooled and dropped, features are extracted and compressed from the original image, and finally the output is completed through the fully connected layer; Step B2: Identify the pointing angle of the plum blossom needle of the water meter; Firstly, the plum blossom needle image is preprocessed using adaptive threshold segmentation technology, and the threshold is dynamically adjusted according to the illumination and texture characteristics of each small area of ​​the image to separate the plum blossom needle from the background. The segmented image is further subjected to line detection through probabilistic Hough line transform. During the detection process, a corresponding algorithm is designed to exclude the wrongly detected lines and only retain the line information that matches the plum blossom needle feature. Finally, the exact position and direction of the plum blossom needle are determined by analyzing the retained straight line information.

2. The water meter reading recognition method based on deep learning according to claim 1 is characterized in that: The step A comprises: Step A1: Image acquisition and preprocessing, first obtain the original image of the dial and perform preprocessing; Step A2: color feature extraction, by analyzing the color distribution of the dial image, using color feature extraction technology to globally analyze the original image, and preliminarily determine the area where the dial data is located; Step A3: Morphological processing: applying morphological image processing algorithm to the preliminary dial data area obtained by color feature extraction to perform regional fusion and noise removal; Step A4: parameter space transformation and dial data positioning, mapping the processed dial data area into the parameter space, and using the Hough circle transformation method to detect and locate the dial data position; Step A5: Verify and optimize the positioning results, and adjust the algorithm parameters according to actual application requirements to adapt to various image conditions and improve positioning accuracy.

3. The water meter reading recognition method based on deep learning according to claim 2 is characterized in that: In the step A1, the preprocessing includes grayscale conversion and noise filtering of the image; In step A2, a more accurate color analysis is performed using the HSV color space to distinguish between the dial data area and the non-dial data area; In step A3, small noise points in the image are removed through dilation and erosion operations, while the continuity of the dial data area is enhanced; In step A4, the Hough circle transform method accurately determines the position and size of the dial data area through accumulator statistics and threshold judgment; Adjusting the algorithm parameters in step A5 includes adjusting the threshold of the Canny edge detector and the accumulator resolution of the Hough transform.

4. The water meter reading recognition method based on deep learning according to claim 1 is characterized in that: In step B1, the image covers different lighting conditions and background complexity; Data preprocessing includes image cropping, grayscale conversion and normalization; Data augmentation techniques include image rotation, image scaling, and adding random noise and blur effects.

5. The water meter reading recognition method based on deep learning according to claim 1 is characterized in that: The input layer of the neural network model accepts a 32×32 pixel color image; the batch normalization layer normalizes the input image; the convolution layer Conv2D: uses the convolution kernel to extract image features, and the output dimension of this layer is kept at 32×32 and the depth is 32; the maximum pooling layer MaxPooling2D: this layer reduces the dimension of the 32×32 feature map to 8×8, and the depth is still 32; the dropout layer Dropout: adds a dropout layer after each pooling layer to randomly discard the activation values ​​of some neurons with a certain probability; the flattening layer Flatten the multi-dimensional output into one dimension, so that it serves as the input of the fully connected layer. This operation converts the 8×8×32 output into a one-dimensional vector of 2048 units; The final fully connected layer Dense is used to output the final result.

6. The water meter reading recognition method based on deep learning according to claim 1 is characterized in that: In step B2, a corresponding algorithm is designed to exclude the wrongly detected straight lines and only retain the straight line information that matches the plum blossom needle feature. It includes, first merging the straight lines, merging the adjacent straight lines together, then calculating the distance of all the straight lines, all the similar straight lines form different clusters, and then calculating the central straight line of each cluster to obtain the central straight line of each cluster, which represents the position of this cluster, and then searching for two straight lines whose intersection point is close to the center of the image and is nearly vertical and retaining them, these two straight lines are the straight line information of the plum blossom needle.

7. The water meter reading recognition method based on deep learning according to claim 1 is characterized in that: It also includes water meter wheel area reading recognition, which includes the following steps: Step C: Use the convolutional neural network for the task of recognizing the readings of the water meter, and train the corresponding neural network model for the reading recognition of the character wheel area; first, perform data preparation, which includes positioning and cropping the image. In the process of positioning the character wheel area of ​​the water meter, segment the character wheel area in the dial image; eliminate interference to obtain the precise area contained in the outer frame of the character wheel; Image registration is used in image positioning tasks, and an ORB-based image positioning algorithm is proposed to align the target image and the standard image; by detecting feature points in the image and calculating the descriptors of these feature points, the target image and the standard image are aligned based on these descriptors; After completing the alignment operation of the image, the fixed ROI of the image is finally cut to obtain the image data required for subsequent tasks; Use data augmentation to expand the training dataset; The word wheel area reading recognition separates the word wheel area from the background and uses an adaptive threshold binarization method to segment the digits. The adaptive threshold binarization method divides the image into multiple local areas and then applies adaptive threshold binarization to each area. The threshold of each local area is calculated based on the local statistical information of the area, and the adaptive threshold binarization method is used to segment the digits; By combining adaptive threshold segmentation with fixed threshold segmentation, using adaptive threshold and dynamic adjustment of fixed threshold segmentation value, performing threshold segmentation operations twice on the same digital image and overlapping the two images to retain the common value to obtain an overlapping image, and performing a dilation operation on the obtained overlapping image, the processed image can reduce a large amount of noise interference while completely retaining the digital image structure; the digital image after the above processing is used for the training of deep learning neural network model; In the neural network model of wheel area reading recognition, VGG16 is used to extract features from the binary image, and the local features of the numbers are learned to solve the recognition problem of half characters. Each binary image is converted into a feature matrix composed of eigenvalues. The feature matrix and the binary image correspond one to one. Then a fully connected neural network is established, and the corresponding binary image feature matrix is ​​used to train the fully connected neural network to obtain a deep learning neural network model with recognition ability. The reading and recognition of the character wheel area are completed through a deep learning neural network model.

8. The water meter reading recognition method based on deep learning according to claim 7 is characterized in that: The process of the image positioning algorithm based on ORB is as follows: (1) Using the ORB feature detector and descriptor generator, the feature points in the two images are detected and the descriptors are calculated; (2) Use Hamming distance to match and obtain matching point pairs between the two images; (3) Screen out excellent matching point pairs; (4) Find the geometric transformation relationship between the two images based on the matching point pairs; (5) Using the RANSAC algorithm, the homography matrix between the two images is calculated from the matching point pairs; (6) Using the homography matrix to perform perspective transformation on the image to be registered to achieve image alignment; After completing the image alignment, the image is further aligned; the image is recognized again, the position of the predefined reference structure in the image is found and the coordinates of the position point with the highest matching degree are returned, and then template matching is used. Finally, an affine transformation is performed based on the obtained position points to obtain a further calibration image.

9. Application of a water meter reading recognition method based on deep learning in water meter calibration, characterized in that: A water meter reading recognition method based on deep learning comprising any one of claims 1 to 6; In the process of using the plum blossom needle to calibrate the water meter, the result obtained by subtracting two erroneous readings is converted into a result obtained by calculating the number of rotations and angles of the plum blossom needle; the calibration of the water meter is completed by taking an image of the water meter dial at the beginning and end, and comparing the difference between the two magnitude pointer readings and the actual flow rate passing through the water meter during this period.

Images