Water meter number and meter hand identification method based on deep learning

Abstract

The invention relates to a water meter number and meter hand identification method based on deep learning, and belongs to the field of computer vision. According to the method, a target area enhancement module and a feature cross cooperation module are used for recognizing the numbers and the hands of the water meter, and the positioning capacity of the numbers and the hands is improved by enhancing the feature expression of a reading area of the water meter; and the decoder is guided to learn the structured information of the water meter reading, so that the model can efficiently analyze the water meter image and output an accurate reading result. The method effectively improves the automatic recognition capability of the water meter reading through a deep learning method, can adapt to different types of water meter dial plates, maintains high-precision recognition in a complex environment, is suitable for the fields of intelligent meter reading, water affair management, energy consumption monitoring and the like, achieves efficient and precise water meter reading automation, improves the management efficiency, and reduces the labor cost.

Description

Technical Field

[0001] This invention belongs to the field of computer vision, specifically relating to a method for recognizing water meter digits and needles based on deep learning. Background Technology

[0002] In recent years, with the rapid development of my country's social economy, the demand for water management and smart meter reading has been increasing, and the automation and intelligence levels of water supply systems have been continuously improving. Water meters, as crucial devices for measuring water consumption, play a vital role in urban water supply, industrial water use, and smart water systems. Accurately and efficiently reading water meter readings and pointer information is a key aspect of the intelligent upgrading of water management, and is of great significance for improving water resource management efficiency and reducing the cost of manual meter reading. Therefore, designing an efficient and accurate water meter reading and pointer identification system is of great value in promoting the development of smart water management.

[0003] Like an identification tag, each water meter has its own unique metering information. A water meter's reading typically consists of two parts: a digital display and a pointer scale. The digital portion shows the cumulative water consumption, while the pointer indicates more precise water usage through different scale markings. The digital display usually uses a mechanical scrolling or electronic display method, while the pointer points to a specific position on the dial for a more precise reading. The reading area is usually located in the center of the dial, and different types of water meters may use different layouts. Common reading arrangements include purely digital displays, pointer scale displays, and hybrid display modes combining digital and pointer scales.

[0004] Existing water meter digit and needle recognition algorithms mainly have two major problems:

[0005] 1) Most methods focus only on a single type of water meter reading. Existing methods typically identify either mechanical or electronic water meters, assuming ideal shooting conditions and failing to consider adaptability in complex environments. Furthermore, many water meter reading recognition methods rely on scene text detection models. However, in water meter readings, digits and pointers are often spatially separated. This necessitates additional segmentation and matching steps when handling mixed display (digits + pointer) modes, increasing computational complexity and potentially leading to reassembly errors, thus reducing overall recognition accuracy.

[0006] 2) Most methods employ a two-stage recognition strategy. Existing methods typically separate water meter digit detection and needle recognition into two independent stages. After detection, additional post-processing operations such as region cropping and angle correction are required. This approach not only increases computational overhead and affects model running speed but also easily leads to error accumulation due to the lack of constraints between different modules, impacting the final recognition accuracy. Our method, however, treats detection and recognition as a whole, allowing digits and needles to share visual features. Furthermore, it optimizes the model through joint loss calculation, improving the stability and accuracy of recognition. Summary of the Invention

[0007] The purpose of this invention is to provide a deep learning-based method for recognizing water meter digits and pointers. This method achieves automated parsing of water meter readings through an end-to-end network, solving the recognition difficulties caused by blurred digits, pointer occlusion, and lighting conditions in complex environments. This invention employs a joint optimization strategy, effectively utilizing the inherent relationship between the digits and pointer positions to improve overall recognition accuracy. This invention proposes a water meter digit and pointer recognition method composed of a target region enhancement module and a feature cross-coordination module. By strengthening the feature representation of the water meter reading region, it enhances the localization ability of digits and pointers; by guiding the decoder to learn the structured information of the water meter reading, the model can efficiently parse water meter images and output accurate reading results. This invention effectively improves the automatic recognition capability of water meter readings through deep learning methods, adapts to different types of water meter dials, and maintains high-precision recognition in complex environments. It is applicable to fields such as smart meter reading, water management, and energy consumption monitoring, achieving efficient and accurate automated water meter reading, improving management efficiency, and reducing labor costs.

[0008] To achieve the above objectives, the technical solution of the present invention is: a method for recognizing water meter digits and pointers based on deep learning, comprising the following steps:

[0009] Step S1: Collect image datasets for water meter reading recognition and annotate the data;

[0010] Step S2: Construct an end-to-end recognition network model for water meter digits and needles based on deep learning;

[0011] Step S3: Train the recognition network model constructed in step S2 so that it can simultaneously detect water meter digits and meter needle information;

[0012] Step S4: Use the trained end-to-end recognition network model for water meter digits and pointers to infer the input water meter image and output the final reading results of the water meter digits and pointers.

[0013] Furthermore, step S1 specifically includes:

[0014] Step S11: Collect a dataset of water meter images, covering different types of water meters and including various environmental factors;

[0015] Step S12: Use the manual annotation tool labelimg to annotate the digital area and pointer position in the water meter image, including the spatial coordinates of the numbers, the spatial coordinates of the pointer, the pointer angle, and the reading information.

[0016] Step S13: Standardize the labeled data and convert it into a data format suitable for deep learning training;

[0017] Step S14: Perform data augmentation processing.

[0018] Furthermore, in step S11, different types of water meters include mechanical water meters and electronic water meters; various environmental factors include changes in light, dirt obstruction, and viewing angle shift.

[0019] Furthermore, in step S14, the enhancement process includes random rotation, scaling, color adjustment, and noise perturbation.

[0020] Furthermore, step S2 specifically includes:

[0021] Step S21: Construct an end-to-end identification network model for the overall water meter digits and needles. This model consists of five parts: a ResNet-50 backbone network, a target region enhancement module, a Transformer encoder, a feature cross-cooperation module, and a Transformer decoder.

[0022] Step S22: Construct a parallel prediction head, complete the pointer reading and positioning through the output of the ResNet-50 backbone network, and complete the digit recognition and digit boundary point prediction through the output of the Transformer decoder;

[0023] Step S23: Construct the loss function for water meter digit and needle recognition, specifically the instance classification loss L. cls Digital recognition loss L num and boundary loss L bd The overall loss function is expressed as:

[0024]

[0025] In the formula λ cls , λ num and λ bd The weights represent the corresponding loss values, and k represents the candidate prediction instance T. k The index.

[0026] Furthermore, the instance classification loss L cls The focus loss FL is used to weigh the performance of digit recognition, and is calculated as follows:

[0027]

[0028] in, Indicates an indicator function, It is a mapping The image represents α, which indicates the hyperparameter balancing positive and negative samples, and γ, which indicates the focus adjustment parameter. This represents the model's predicted probability that the k-th sample is a positive sample;

[0029] Digital classification loss L num The CTC loss is used to address the length discrepancy between the actual predicted value and the predicted result in numerical prediction. It is calculated as follows:

[0030]

[0031] Among them, t, These represent the target label and the predicted value, respectively. This represents the k-th sample after mapping;

[0032] Boundary loss L bd The center line and the differences between the upper and lower boundary lines and the actual rectangle are calculated using the L1 norm, as follows:

[0033]

[0034] Where top represents the sampling points of the upper boundary line, bot represents the sampling points of the lower boundary line, N represents the number of prediction boxes, and n represents the index of the prediction box currently being calculated.

[0035] Furthermore, step S3 specifically includes:

[0036] Step S31: Initialize the parameters of the water meter digit and needle detection and recognition network using a random number seed;

[0037] Step S32: The initial learning rate is set to 0.001, and a dynamic learning rate adjustment method is adopted. The first 5000 iterations are trained with a fixed initial learning rate, and the next 25000 iterations follow the "Poly" multi-learning rate strategy.

[0038] Step S33: Perform preprocessing on the input water meter image, normalize the image input size to 512×512, and then apply various image enhancement methods.

[0039] Step S34: Use the water meter's digital information, pointer information, and location information as supervisory information to train the preprocessed image;

[0040] Step S35: Save the trained end-to-end recognition network model framework for water meter digits and needles and the trained weights of the end-to-end recognition network model for water meter digits and needles on the server platform.

[0041] Furthermore, in step S33, the image enhancement method includes random rotation, random scaling, and random horizontal flipping.

[0042] Furthermore, step S4 specifically includes:

[0043] Step S41: Load the trained end-to-end recognition network model for water meter digits and needles, along with its parameters;

[0044] Step S42: Preprocess the input water meter image;

[0045] Step S43: Using the rectangular box query mechanism, predict the water meter digital position, water meter digital reading, meter needle position, and meter needle reading in the prediction head after the ResNet-50 backbone network and Transformer decoder, and realize position detection and reading recognition by fitting the detection area with a rectangular box.

[0046] Step S44: Combine the prior information of the water meter type, normalize the meter needle angle, and calculate the final water meter reading based on the digital detection results.

[0047] Furthermore, in step S42, the preprocessing operations for the input water meter image include: color normalization, rotation correction, and scale transformation.

[0048] Compared with the prior art, the present invention has the following beneficial effects:

[0049] 1. This invention achieves automated parsing of water meter readings through an end-to-end network, solving the identification difficulties caused by blurred water meter numbers, obstructed meter needles, and lighting effects in complex environments.

[0050] 2. This invention captures the relative characteristics between the water meter digits and the pointer, and designs detection and recognition branches in a targeted manner, making it more adaptable to the detection and recognition tasks of water meter digits and pointers.

[0051] 3. This invention proposes a water meter digit and pointer recognition method guided by a target area enhancement module, which improves the ability to locate digits and pointers by strengthening the feature representation of the water meter reading area.

[0052] 4. This invention proposes a water meter digit and needle recognition method guided by a feature cross-cooperative module. By guiding the decoder to learn the structured information of the water meter reading, the model can efficiently parse the water meter image and output accurate reading results. Attached Figure Description

[0053] Figure 1 This is a network structure block diagram of an example of the present invention;

[0054] Figure 2 This is a structural diagram of the target region enhancement module in an example of the present invention;

[0055] Figure 3 This is an example diagram of a water meter detection and identification tag according to an embodiment of the present invention;

[0056] Figure 4 This is an example diagram of the water meter dataset involved in the embodiments of the present invention;

[0057] Figure 5 This is an example diagram showing the detection and identification results of the water meter digits and needles in an embodiment of the present invention;

[0058] Figure 6 This is a network data diagram for detecting and recognizing water meter digits and needles, as an example of the present invention.

[0059] Figure 7 This is a convergence curve of the accuracy (IOU = 0.5) of the water meter digit and needle detection and recognition network structure in this invention example;

[0060] Figure 8 This is a convergence curve of the loss function of the water meter digit and needle detection and recognition network structure in an example of the present invention.

[0061] Figure 9 This is a demonstration of the recognition effect on the APP side of this invention. Detailed Implementation

[0062] The technical solution of the present invention will now be described in detail with reference to the accompanying drawings.

[0063] This invention provides a deep learning-based method for recognizing water meter digits and pointers, comprising the following steps:

[0064] Step S1: Collect an image dataset for water meter reading recognition and annotate the data;

[0065] Step S2: Construct an end-to-end recognition network model for water meter digits and needles based on deep learning;

[0066] Step S3: Train the recognition network model constructed in step S2 so that it can simultaneously detect water meter digits and meter needle information;

[0067] Step S4: Use the trained end-to-end recognition network model for water meter digits and pointers to infer the input water meter image and output the final reading results of the water meter digits and pointers.

[0068] The following is a detailed implementation process of the present invention.

[0069] This invention discloses a deep learning-based method for recognizing water meter digits and pointers, consisting of a target region enhancement module and a feature cross-coordination module. By strengthening the feature representation of the water meter reading region, it improves the localization capability of digits and pointers. By guiding the decoder to learn the structured information of the water meter reading, the model can efficiently parse water meter images and output accurate reading results.

[0070] like Figure 1 As shown, this embodiment provides a method for recognizing water meter digits and pointers based on deep learning. Figure 1 The overall network framework diagram of this method, and the specific implementation of the invention, includes the following steps:

[0071] Step S1: Collect image datasets for water meter reading recognition and annotate the data;

[0072] Step S2: Construct an end-to-end recognition network for water meter digits and needles based on deep learning;

[0073] Step S3: Train the recognition network so that it can simultaneously detect the water meter readings and the needle information;

[0074] Step S4: Use the trained model to infer the input water meter image and output the final reading results of the water meter digits and the pointer.

[0075] In this embodiment, step S1 specifically includes the following steps:

[0076] Step S11: Collect water meter image datasets, covering different types of water meters (such as mechanical water meters and electronic water meters), and including various environmental factors (such as changes in lighting, dirt obstruction, and viewing angle shift).

[0077] Step S12: Use the manual annotation tool labelimg to annotate the digital area and pointer position in the water meter image, including the spatial coordinates of the numbers and pointer, pointer angle and reading information, etc.

[0078] Step S13: Standardize the labeled data and convert it into a data format suitable for deep learning training so that it can be used for subsequent model training;

[0079] Step S14: Perform data augmentation processing, including random rotation, scaling, color adjustment, and noise perturbation, to improve the robustness and generalization ability of the model.

[0080] In this embodiment, step S2 specifically includes the following steps:

[0081] Step S21: Construct an overall water meter digit and needle recognition network model. This model consists of five parts: ResNet-50 backbone network, target area enhancement module, Transformer encoder, feature cross-cooperation module, and Transformer decoder.

[0082] Step S22: Construct a parallel prediction head. The output of the ResNet-50 backbone network is used to complete the meter needle reading and positioning. The output of the Transformer decoder is used to complete the digit recognition and digit boundary point prediction, thereby improving the accuracy of water meter readings.

[0083] Step S23: Construct the loss function for water meter digit and needle recognition, specifically the instance classification loss L. cls Digital recognition loss L num and boundary loss L bd The overall loss function is expressed as:

[0084]

[0085] In the formula λ cls , λ num and λ bd The weights represent the corresponding loss values, and k represents the candidate prediction instance T. k The index.

[0086] Instance classification loss L cls The focus loss FL is used to weigh the performance of digit recognition, and is calculated as follows:

[0087]

[0088] in, Indicates an indicator function, It is a mapping The image represents α, which indicates the hyperparameter balancing positive and negative samples, and γ, which indicates the focus adjustment parameter. This represents the model's predicted probability that the k-th sample is a positive sample;

[0089] Digital classification loss L num The CTC loss is used to address the length discrepancy between the actual predicted value and the predicted result in numerical prediction. It is calculated as follows:

[0090]

[0091] Among them, t, These represent the target label and the predicted value, respectively. This represents the k-th sample after mapping;

[0092] Boundary loss L bdThe center line and the differences between the upper and lower boundary lines and the actual rectangle are calculated using the L1 norm, as follows:

[0093]

[0094] Where top represents the sampling points of the upper boundary line, bot represents the sampling points of the lower boundary line, N represents the number of prediction boxes, and n represents the index of the prediction box currently being calculated.

[0095] Furthermore, step S21 specifically includes the following steps:

[0096] Step S211, as follows Figure 2 As shown, the target region enhancement module divides the input features into G sub-features, denoted as F = [F0, F1] [F2] [F3] [F4] [F5] [F6] [F7] [F8] [F9] [F1] [F1] [F2 ...2] [F3] [ i ,...,F G-1 Then, the target region features are highlighted through global feature enhancement. The calculation formula is as follows:

[0097] F S1 =Softmax(Conv 3×3 (F i ))

[0098] F A1 =GAP(Conv 3×3 (F i ))

[0099] Among them, F S1 To perform global features in the channel dimension C using Softmax, F A1 For global features that are globally averaged pooled in spatial dimensions H and W, Conv 3×3 This is a convolutional layer with a kernel size of 3*3, where Softmax is the activation function and GAP is the global average pooling.

[0100] Secondly, edge features of the dial and hands are extracted through local feature enhancement. The calculation formula is as follows:

[0101] F X =F Y =Conv 1×1 (Cat(Xavg(F i ),Yavg(F i )))

[0102] Among them, F X Let F be the eigenvector in the X direction. Y Let be the feature vector in the Y direction, Cat denotes the concatenation operation, Xavg denotes the average pooling along the horizontal direction, and Yavg denotes the average pooling along the vertical direction.

[0103] Finally, the enhanced features of the target region are obtained by fusing global and local features, calculated as follows:

[0104] F = Weight(Mat(F) S1 ,F A2 ),F XY )

[0105] Here, Weight represents the weighted operation.

[0106] Step S212: The feature cross-coordination module, by leveraging the features of the interactive digits and the pointer, and utilizing the continuity of the digits and their relative positions, selects highly reliable candidate regions and recognition results, thereby improving the accuracy of water meter digit and pointer detection and recognition. The calculation is as follows:

[0107] N K =Sel(P i N num ×N cls )

[0108] Among them, P i This represents all predictions obtained through the Transformer encoder. Sel is the operation that selects the prediction with the highest confidence. N num For the confidence level of the water meter reading prediction, N cls Classify the confidence level of the water meter pointer.

[0109] Furthermore, step S3, training the deep learning-based water meter digit and pointer recognition network, specifically involves:

[0110] Step S31: Initialize the parameters of the water meter digit and needle detection and recognition network using a random number seed;

[0111] Step S32: The initial learning rate is set to 0.001, and a dynamic learning rate adjustment method is adopted. The first 5000 iterations are trained with a fixed initial learning rate, and the next 25000 iterations follow the "Poly" multi-learning rate strategy.

[0112] Step S33: Perform preprocessing on the input water meter image, normalize the image input size to 512×512, and then perform various image enhancement methods, such as random rotation, random scaling, and random horizontal flipping.

[0113] Step S34: Use the water meter's digital information, pointer information, and location information as supervisory information to train the preprocessed image;

[0114] Step S35: Save the trained water meter digit and pointer detection and recognition framework and the trained water meter digit and pointer detection and recognition network weights to the server platform.

[0115] Furthermore, step S4 specifically includes:

[0116] Step S41: Load the trained water meter digit and pointer recognition model and its parameters;

[0117] Step S42: Preprocess the input water meter image, including color normalization, rotation correction, and scale transformation;

[0118] Step S43: Using the rectangular box query mechanism, predict the position of the water meter digits, the water meter digit reading, the position of the meter needle, and the meter needle reading in the prediction head after the backbone network ResNet-50 and the Transformer decoder, and realize position detection and reading recognition by fitting the detection area with a rectangular box.

[0119] Step S44: Combine the prior information of the water meter type, normalize the meter needle angle, and calculate the final water meter reading based on the digital detection results.

[0120] Figure 3 This is an example diagram of a water meter detection and identification tag according to an embodiment of the present invention; Figure 4 This is an example diagram of the water meter dataset involved in the embodiments of the present invention; Figure 5 This is an example diagram showing the detection and identification results of the water meter digits and needles in an embodiment of the present invention; Figure 6 This is a network data diagram for detecting and recognizing water meter digits and needles, as an example of the present invention. Figure 7 This is a convergence curve of the accuracy (IOU = 0.5) of the water meter digit and needle detection and recognition network structure in this invention example; Figure 8 This is a convergence curve of the loss function of the water meter digit and needle detection and recognition network structure in an example of the present invention. Figure 9 This is a demonstration of the recognition effect on the APP side of this invention.

[0121] The above are preferred embodiments of the present invention. Any changes made to the technical solution of the present invention that do not exceed the scope of the technical solution of the present invention shall fall within the protection scope of the present invention.

Claims

1. A method for recognizing water meter digits and needles based on deep learning, characterized in that, Includes the following steps: Step S1: Collect an image dataset for water meter reading recognition and annotate the data; Step S2: Construct an end-to-end recognition network model for water meter digits and needles based on deep learning; Step S3: Train the recognition network model constructed in step S2 so that it can simultaneously detect water meter digits and meter needle information; Step S4: Use the trained end-to-end recognition network model for water meter digits and pointers to infer the input water meter image and output the final reading results of the water meter digits and pointers.

2. The method for recognizing water meter digits and needles based on deep learning according to claim 1, characterized in that, Step S1 specifically includes: Step S11: Collect a dataset of water meter images, covering different types of water meters and including various environmental factors; Step S12: Use the manual annotation tool labelimg to annotate the digital area and pointer position in the water meter image, including the spatial coordinates of the numbers, the spatial coordinates of the pointer, the pointer angle, and the reading information. Step S13: Standardize the labeled data and convert it into a data format suitable for deep learning training; Step S14: Perform data augmentation processing.

3. The method for recognizing water meter digits and needles based on deep learning according to claim 2, characterized in that, In step S11, different types of water meters include mechanical water meters and electronic water meters; various environmental factors include changes in light, dirt obstruction, and viewing angle shift.

4. The method for recognizing water meter digits and needles based on deep learning according to claim 2, characterized in that, In step S14, the enhancement process includes random rotation, scaling, color adjustment, and noise perturbation.

5. The method for recognizing water meter digits and needles based on deep learning according to claim 1, characterized in that, Step S2 specifically includes: Step S21: Construct an end-to-end identification network model for the overall water meter digits and needles. This model consists of five parts: a ResNet-50 backbone network, a target region enhancement module, a Transformer encoder, a feature cross-cooperation module, and a Transformer decoder. Step S22: Construct a parallel prediction head, complete the pointer reading and positioning through the output of the ResNet-50 backbone network, and complete the digit recognition and digit boundary point prediction through the output of the Transformer decoder; Step S23: Construct the loss function for water meter digit and needle recognition, specifically the instance classification loss L. cls Digital recognition loss L num and boundary loss L bd The overall loss function is expressed as: In the formula λ cls , λ num and λ bd The weights represent the corresponding loss values, and k represents the candidate prediction instance T. k The index.

6. The method for recognizing water meter digits and pointers based on deep learning according to claim 5, characterized in that, Instance classification loss L cls The focus loss FL is used to weigh the performance of digit recognition, and is calculated as follows: in, Indicates an indicator function, It is a mapping The image represents α, which indicates the hyperparameter balancing positive and negative samples, and γ, which indicates the focus adjustment parameter. This represents the model's predicted probability that the k-th sample is a positive sample; Digital classification loss L num The CTC loss is used to address the length discrepancy between the actual predicted value and the predicted result in numerical prediction. It is calculated as follows: Among them, t, These represent the target label and the predicted value, respectively. This represents the k-th sample after mapping; Boundary loss L bd The center line and the differences between the upper and lower boundary lines and the actual rectangle are calculated using the L1 norm, as follows: Where top represents the sampling points of the upper boundary line, bot represents the sampling points of the lower boundary line, N represents the number of prediction boxes, and n represents the index of the prediction box currently being calculated.

7. The method for recognizing water meter digits and needles based on deep learning according to claim 1, characterized in that, Step S3 specifically includes: Step S31: Initialize the parameters of the water meter digit and needle detection and recognition network using a random number seed; Step S32: The initial learning rate is set to 0.001, and a dynamic learning rate adjustment method is adopted. The first 5000 iterations are trained with a fixed initial learning rate, and the next 25000 iterations follow the "Poly" multi-learning rate strategy. Step S33: Perform preprocessing on the input water meter image, normalize the image input size to 512×512, and then apply various image enhancement methods. Step S34: Use the water meter's digital information, pointer information, and location information as supervisory information to train the preprocessed image; Step S35: Save the trained end-to-end recognition network model framework for water meter digits and needles and the trained weights of the end-to-end recognition network model for water meter digits and needles on the server platform.

8. The method for recognizing water meter digits and pointers based on deep learning according to claim 7, characterized in that, In step S33, the image enhancement method includes random rotation, random scaling, and random horizontal flipping.

9. The method for recognizing water meter digits and pointers based on deep learning according to claim 5, characterized in that, Step S4 specifically includes: Step S41: Load the trained end-to-end recognition network model for water meter digits and needles, along with its parameters; Step S42: Preprocess the input water meter image; Step S43: Using the rectangular box query mechanism, predict the water meter digital position, water meter digital reading, meter needle position, and meter needle reading in the prediction head after the ResNet-50 backbone network and Transformer decoder, and realize position detection and reading recognition by fitting the detection area with a rectangular box. Step S44: Combine the prior information of the water meter type, normalize the meter needle angle, and calculate the final water meter reading based on the digital detection results.

10. A method for recognizing water meter digits and pointers based on deep learning according to claim 9, characterized in that, In step S42, the preprocessing operations for the input water meter image include: color normalization, rotation correction, and scale transformation.

Images