A water meter reading recognition method

Abstract

The application discloses a kind of water meter reading identification methods, comprising the following steps: S1: water meter image is collected by image acquisition module, wherein, image acquisition module includes controller, remote transmission module and camera module, each water meter is equipped with image acquisition module, water meter is digital water meter or pointer type water meter, water meter image is digital water meter image or pointer type water meter image;S2: digital water meter image is uploaded to first water meter reading identification system, or, pointer type water meter image is uploaded to second water meter reading identification system;S3: reading identification is carried out to digital water meter image using first water meter reading identification system, or, reading identification is carried out to pointer type water meter image using second water meter reading identification system.The application can realize the automatic identification of water meter reading;Realize the reading identification of corresponding water meter image of different types;The installation of image acquisition module does not need to cut off water, and the installation cost is low, which is suitable for popularization and use.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of water meter reading, in particular to a water meter reading recognition method. BACKGROUND

[0002] Water meter reading recognition is an important link in the process of urban water management. With the continuous expansion of the city scale, the amount of information of city management is also increasing, at this time, the intelligent degree of city information management needs to be continuously improved, and both city information management and intelligent life of residents cannot be separated from the management and utilization of water resources. Intelligent management of water resources cannot be separated from intelligent management of water meters. Water meters play an essential role in water resource management and play an important role in the development of the entire digital city. Water meters have a wide range of applications in industry, life and other fields.

[0003] Among them, the water meter usually includes a digital water meter or a pointer type water meter. For water meter reading, the traditional method is to manually read the meter. This manual meter reading method has a large workload, low efficiency, long time consumption and high labor cost, and the staff is prone to errors when processing a large amount of data, so a new water meter reading recognition method is needed. SUMMARY

[0004] (I) Technical problems to be solved

[0005] In view of the deficiencies in the prior art, the present application provides a water meter reading recognition method, which can solve the above technical problems.

[0006] (II) Technical solutions

[0007] To solve the above technical problems, the present application provides the following technical solutions: a water meter reading recognition method, comprising the following steps:

[0008] S1: acquiring a water meter image through an image acquisition module, wherein the image acquisition module includes a controller, a remote transmission module and a camera module, each water meter is provided with an image acquisition module, the water meter is a digital water meter or a pointer type water meter, and the water meter image is a digital water meter image or a pointer type water meter image;

[0009] S2: uploading the digital water meter image to a first water meter reading recognition system, or uploading the pointer type water meter image to a second water meter reading recognition system;

[0010] S3: using the first water meter reading recognition system to recognize the reading of the digital water meter image, or using the second water meter reading recognition system to recognize the reading of the pointer type water meter image.

[0011] Preferably, when the water meter image is a digital water meter image, step S3 corresponds to the following substeps:

[0012] S31: detecting a counter frame of the digital water meter image to obtain a counter frame image;

[0013] S32: performing corner point detection on the counter frame image to obtain corner point positions;

[0014] S33: correcting the counter frame image using the corner point positions;

[0015] S34: performing character recognition on the counter frame image.

[0016] Preferably, the first water meter reading recognition system is provided with a counter frame detection model, and the sub-step S31 is detecting a counter frame of the digital water meter image using the counter frame detection model.

[0017] Preferably, the counter frame detection model adopts an improved YOLOv7 model.

[0018] Preferably, the sub-step S32 further includes readable / unreadable classification of the counter frame image; further, the subsequent sub-step S33 is performed on the counter frame image classified as readable.

[0019] Preferably, the first water meter reading recognition system is provided with a multi-task network, and the sub-step S32 specifically includes performing corner point detection and readable / unreadable classification on the counter frame image using the multi-task network.

[0020] Preferably, the counter frame image includes four corner points: a top-left corner, a bottom-left corner, a top-right corner and a bottom-right corner, and the sub-step 33 specifically includes correcting the counter frame image by a perspective transformation: the corner point positions include x and y coordinates corresponding to the corner points, the four corner points of the counter frame image before correction are referred to as p1…p4, and the four corner points after correction are referred to as p′1…p′4, p′1 and p′2 are respectively located at the corner positions of the top-left corner and the top-right corner, and the coordinates of p′3 are determined by the following formulas (1) and (2):

[0021] x′3=x′1 (1)

[0022]

[0023] In the above formula (2), [p m , p n ] dist is the Euclidean distance between p m , p n ; further x p′4 is determined by x′4=x′2 and y′4=y′3.

[0024] Preferably, the first water meter reading recognition system is provided with a character recognition model, and the sub-step S34 is performing character recognition on the counter frame image using the character recognition model.

[0025] Preferably, the character recognition model adopts an improved YOLOv7 model.

[0026] Preferably, when the water meter image is a pointer type water meter image, step S3 corresponds to: using a second water meter reading recognition system to perform dial plate segmentation positioning on the pointer type water meter image to obtain the positions of the centers of each sub-dial plate; determining the order of each sub-dial plate according to the positions of the centers of each sub-dial plate; and obtaining the pointer reading of each sub-dial plate in order.

[0027] (Three) beneficial effects

[0028] Compared with the prior art, the present application provides a water meter reading recognition method, which has the following beneficial effects: (1) by using the first water meter reading recognition system to recognize the reading of the digital water meter image, and using the second water meter reading recognition system to recognize the reading of the pointer type water meter image, the automatic recognition of the water meter reading is realized, which can improve the work efficiency and reduce the labor cost compared with the traditional manual meter reading method, and better avoid the error of water meter reading caused by human factors; (2) different water meter reading recognition systems are used to recognize the corresponding reading of different types of water meter images, realizing the classification recognition processing of the water meter reading to improve the recognition accuracy; (3) by installing an image acquisition module composed of a controller, a camera module and a remote transmission module on the water meter, timed photographing and uploading can be completed, and the installation does not need to cut off the water, the installation cost is low, and it is suitable for popularization and use. BRIEF DESCRIPTION OF DRAWINGS

[0029] Figure 1 is a step flow chart of the water meter reading recognition method of the present application;

[0030] Figure 2 is a network structure diagram of the improved YOLOv7 model of the present application;

[0031] Figure 3 is a schematic diagram of the present application for correcting the predicted corner points by transmission transformation;

[0032] Figure 4 is a structure diagram of the multi-task network of the present application;

[0033] Figure 5 is a schematic diagram of the counter frame image of the present application;

[0034] Figure 6 is a principle block diagram of the image acquisition module of the present application. DETAILED DESCRIPTION

[0035] With reference to the drawings and the embodiments of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments of the present application, all the other embodiments obtained by those skilled in the art without creative work fall within the scope of the present application.

[0036] The present application provides a water meter reading recognition method, comprising the following steps:

[0037] S1: acquiring a water meter image through an image acquisition module, wherein the image acquisition module comprises a controller 1, a remote transmission module 2 and a camera module 3, each water meter is provided with an image acquisition module, the water meter is a digital water meter or a pointer type water meter, and the water meter image is a digital water meter image or a pointer type water meter image.

[0038] The digital water meter is a type of water meter such as a word wheel type water meter, and correspondingly, the digital water meter image comprises a counter for indicating the water meter reading, and one counter comprises a plurality of word wheel frames of different ranges, and each word wheel frame has different word wheel readings of 0-9.

[0039] S2: uploading the digital water meter image to a first water meter reading recognition system, or uploading the pointer type water meter image to a second water meter reading recognition system. It should be understood that the above controller is connected with the remote transmission module and the camera module, and the controller controls the remote transmission module to upload the water meter image collected by the camera module to the first water meter reading recognition system or the second water meter reading recognition system.

[0040] S3: using the first water meter reading recognition system to recognize the reading of the digital water meter image, or using the second water meter reading recognition system to recognize the reading of the pointer type water meter image.

[0041] Specifically, when the water meter image is a digital water meter image, step S3 corresponds to the following sub-steps:

[0042] S31: detecting the counter frame of the digital water meter image to obtain a counter frame image.

[0043] The first water meter reading recognition system is provided with a counter frame detection model, and sub-step S31 is to detect the counter frame of the digital water meter image by using the counter frame detection model. Preferably, before sub-step S31, the following steps can also be included: pre-processing the digital water meter image to screen out images with good quality, clear counters, and covering different angles, lighting conditions and environmental backgrounds as much as possible. Further, an image labeling tool can be used to label the collected digital water meter image, and the position of each counter is labeled with a frame to ensure that the frame closely surrounds the counter. Then, the labeled image is subjected to data enhancement to expand the data set, and rotation, translation, scaling, flipping and other operations are used to increase the diversity and robustness of the data. At the same time, it is also considered to use operations such as blurring and brightness adjustment to simulate changes under different environmental conditions. Finally, the data set can be divided into a training set, a validation set and a test set according to a ratio of 7:2:1; further, the counter frame detection model can be trained by using the data set, so as to detect the counter frame of the digital water meter image by using the trained counter frame detection model.

[0044] Further, the counter frame detection model adopts an improved YOLOv7 model: as shown in Figure 2 The improved YOLOv7 model uses E-ELAN (Extended Efficient Layer Aggregation Network), connection-based model scaling and reparameterization to achieve a favorable balance between detection efficiency and accuracy. The improved YOLOv7 model adopts Mosaic and mixed data enhancement techniques in the preprocessing part of the input, and uniformly scales the image to 640x640 size to input the backbone network. The backbone network is composed of three main components: CBS, E-ELAN and MP1, and the CBAM attention mechanism is introduced in the backbone network and the head network to capture spatial and channel correlations, improve the network feature extraction capability, and avoid interference from other background features; the regression prediction adopts IoU Loss, and the classification prediction adopts BCE Loss to train the network and generate the corresponding weight file, wherein IoU is the intersection over union of the predicted frame and the real frame, and the BCE Loss (Binary Cross-Entropy Loss) loss function is as follows:

[0045] BCE Loss = -(y_true * log(y_pred) + (1-y_true) * log(1-y_pred))

[0046] In the above formula, y_true represents the true label, y_pred represents the output predicted by the model, log represents the natural logarithm function, and * represents element-wise multiplication between elements.

[0047] It should be understood that after the digital water meter image goes through the above-mentioned sub-step S31, the accurate position coordinates of the counter of the digital water meter detected in the picture can be recognized to crop the image of the counter, i.e. the counter frame image, in the digital water meter image. Preferably, after obtaining the counter frame image, the counter frame image is further edge filled, i.e. the frame of the counter frame image is expanded by 10% in a zero padding manner, which facilitates subsequent corner point detection.

[0048] S32: corner point detection is performed on the counter frame image to obtain the corner point position.

[0049] Preferably, the sub-step S32 further comprises readable / unreadable classification of the counter frame image; further, the subsequent sub-step S33 is performed on the counter frame image classified as readable, i.e. the unreadable counter frame image is avoided to enter the subsequent correction and character recognition stage.

[0050] S33: correction of the counter frame image by using the corner point position.

[0051] The counter frame image comprises four corner points: the upper left corner, the lower left corner, the upper right corner and the lower right corner. The sub-step 33 specifically comprises correction of the counter frame image by using a perspective transformation: the corner point position comprises the x and y coordinates of the corner points. The four corner points of the counter frame image before correction are referred to as p1...p4 (i.e. the corner point position obtained in the above-mentioned sub-step S32), and the four corner points of the counter frame image after correction are referred to as p'1...p'4, as shown in the following formula (3): Figure 3 Mapping the corner points predicted in the above-mentioned sub-step S32 to the coordinate space after correction will assume that the image after correction has a fixed size, and p'1 and p'2 are located at the upper left corner and the upper right corner of the image after correction, respectively, i.e. the values of p'1 and p'2 can be calculated in advance: for example, for an image with a size of 320x320, it is assumed that the size of the image after correction will not change, and the positions of p'1 and p'2 are (10, 10) and (310, 10), respectively. Here, a certain number of pixels are reserved to avoid loss of information at the edge of the image during transformation, which affects the correction result. Further, the coordinates of p'3 are determined by the following formula (1) and (2):

[0052] x'3=x'1 (1)

[0053]

[0054] In the above formula (2), [p m , p n ] dist is p m , p nEuclidean distance between p'1 and p'2, Wrct represents the width of the corrected counter frame image, i.e. the distance of p'1 and p'2 on the x-axis; p'4 is further determined by x'4=x'2 and y'4=y'3. It should be understood that the coordinate information of the four corner points p'1...p'4 is obtained, i.e. the correction of the counter frame image is realized.

[0055] Specifically, the first water meter reading recognition system is provided with a multi-task network, and the sub-step S32 specifically comprises angle point detection and readable / non-readable classification of the counter frame image by using the multi-task network: by using a multi-task network, the counter frame image is analyzed and nine outputs are predicted: eight floating-point numbers representing the positions of the four corner points, and an array containing two floating-point numbers representing the probability of the readability of the counter frame, and the multi-task network structure is as shown in Figure 4 Figure 4 In the formula, conv represents convolution, and max represents maxpooling.

[0056] Further, the counter frame image is classified as readable / non-readable: the array composed of two floating-point numbers representing the relative probability of readability or unreadability can be specifically predicted by using a softmax activation function, and the sum of the two floating-point numbers is 1; by setting a threshold for the probability, the fault warning of the water meter is realized, for example, if the probability of unreadability exceeds the preset threshold, the counter frame image corresponds to unreadability, so as to screen out unreadable counter frame images to avoid entering the subsequent correction and character recognition stage, thereby saving the recognition time.

[0057] Specifically, in addition to the coordinates of the above-mentioned corner points, classification labels of readability and unreadability are added in the data set image samples, so as to distinguish normal water meter images and unreadable water meter images caused by shielding and the like. The multi-task network described above learns the features and patterns of different counter frame samples in the training process, and can give the probability values of readability and unreadability of each counter frame sample; in the multi-task network Figure 4 In the multi-task network, there are two non-shared fully connected layers for each output, and in the second non-shared fully connected layer, there are two units for predicting the readability of the counter frame, and a softmax activation function is used to normalize the output of the prediction result and convert it into a value between 0 and 1 as the probability distribution of readability / non-readability, and the sum of the two values is 1, so as to ensure that the sum of the probabilities is 1. For each counter frame image sample, if the probability output by the readability unit is high, it means that the multi-task network considers that the counter frame is readable, otherwise, it means that the counter frame is unreadable, thereby realizing the classification task of the readability / non-readability of the counter frame.

[0058] S34: performing character recognition on the counter frame image.

[0059] ​Specifically, the first water meter reading recognition system is provided with a character recognition model, and the sub-step S34 is to perform character recognition on the counter frame image by using the character recognition model, and the object of recognition detection is the numeral character of the dial from 0 to 9. Preferably, the character recognition model can adopt the improved YOLOv7 model as in the above sub-step S31.

[0060] In addition, in the above sub-step S34, when two characters are recognized to appear in the same dial frame of the counter frame image, that is, one character is above and one character is below in the same dial frame (two incomplete characters, that is, the problem of half characters commonly appearing in dial characters), a dynamic optimization strategy is adopted to adjust the recognized character reading: at this time, the areas of the two characters (specifically, the areas of the bounding boxes of the two characters respectively) are calculated, and when the area proportion of one of the characters in the dial frame exceeds a preset threshold, it is the character reading of the current dial frame: for example Figure 5 As shown in the figure, two characters appear in the 5th dial frame from left to right: the upper character 4 and the lower character 5, and at this time, it is assumed that the area proportion of the lower character 5 in the 5th dial frame exceeds the preset threshold of 70%, then the character reading of the current dial frame (the 5th dial frame) corresponds to 5.

[0061] In addition, in the above sub-step S34, when the area proportions of the two characters in the dial frame do not exceed the preset threshold (for example, the area proportion of the upper character in the dial frame is 40%, the area proportion of the lower character in the dial frame is 60%, and the preset threshold is 70%), then further adjustment is made according to the character reading of the dial frame of the next range (from left to right, the dial frame adjacent to the right side of the current dial frame is the dial frame of the next range): if the character reading of the dial frame of the next range is 9, then the character reading of the current dial frame is the upper character of the two characters; if the character reading of the dial frame of the next range is 0, then the character reading of the current dial frame is the lower character of the two characters.

[0062] In addition, the pointer type water meter image includes a plurality of sub-dials, each of which is arranged in sequence and corresponds to a different range, and each of which has a pointer for indicating the reading. When the water meter image is a pointer type water meter image, step S3 corresponds to: using the second water meter reading identification system to perform dial segmentation positioning on the pointer type water meter image to obtain the positions of the centers of each sub-dial; determining the order of each sub-dial according to the positions of the centers of each sub-dial, which can specifically calculate the angle according to the vector formed by connecting the position coordinates of the center of each sub-dial to determine the pointer rotation angle of each sub-dial and the order of each sub-dial; and obtaining the pointer reading of each sub-dial in sequence, which can specifically obtain the pointer reading of each sub-dial in sequence by using a neural network classification method. Further, the dial reading can be post-processed according to the prior information obtained before; and the reading can be adjusted according to the prior information of the ranges of the front and rear dials to solve the problem of identification error when the pointer is at the critical position; and finally, the final reading result is generated.

[0063] Compared with the prior art, the present application provides a water meter reading identification method, which has the following beneficial effects: (1) by using the first water meter reading identification system to identify the reading of the digital type water meter image and using the second water meter reading identification system to identify the reading of the pointer type water meter image, the automatic identification of the water meter reading is realized, which can improve the work efficiency and reduce the labor cost compared with the traditional manual meter reading method, and better avoid the error of water meter reading caused by human factors; (2) different water meter reading identification systems are used to identify the reading of different types of water meter images respectively, which realizes the classification identification processing of the water meter reading to improve the identification accuracy; (3) by installing an image acquisition module composed of a controller, a camera module and a remote transmission module on the water meter, timed photographing and uploading can be completed, and the installation does not require water interruption, the installation cost is low, and it is suitable for popularization and use; (4) the present application uses a dynamic optimization strategy to improve the problem of half-character identification that is prone to occur in character recognition of digital type water meters such as dial type water meters, thereby ensuring the accuracy of the water meter reading; (5) by using a multi-task network to perform corner point detection and fault warning tasks on the counter frame image, the rotation and distortion dial can be corrected, which can provide help for subsequent character recognition, realize more stable reading, and can filter out unreadable water meters to avoid entering the character detection stage, thereby saving identification time.

[0064] It should be noted that the terms "comprising", "including", or any other variant thereof are intended to cover non-exclusive inclusions, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or includes elements inherent to such a process, method, article or device. Without more limitations, the element defined by the statement "including a" does not exclude the presence of additional identical elements in the process, method, article or device including the element.

[0065] While embodiments of the application have been shown and described, it is to be understood that the application is not limited to the details of the embodiments described, since numerous changes, modifications, substitutions and variations can be made thereto without departing from the spirit and scope of the application as defined by the appended claims and their equivalents.

Claims

1. A water meter reading recognition method, characterized by, The method comprises the following steps: S1: acquiring a water meter image by an image acquisition module, wherein the image acquisition module comprises a controller, a remote transmission module and a camera module, each water meter is provided with the image acquisition module, the water meter is a digital water meter or a pointer type water meter, and the water meter image is a digital water meter image or a pointer type water meter image; S2: uploading the digital water meter image to a first water meter reading identification system or uploading the pointer type water meter image to a second water meter reading identification system; S3: identifying the reading of the digital water meter image by the first water meter reading identification system or identifying the reading of the pointer type water meter image by the second water meter reading identification system; When the water meter image is a digital water meter image, the step S3 comprises the following sub-steps: S31: detecting a counter frame of the digital water meter image to obtain a counter frame image; S32: detecting an angle point of the counter frame image to obtain an angle point position; S33: correcting the counter frame image by using the angle point position; S34: identifying a character of the counter frame image; The sub-step S32 further comprises classifying the counter frame image as readable or unreadable; further, the sub-step S33 is executed on the counter frame image classified as readable; The first water meter reading identification system is provided with a multi-task network, and the sub-step S32 specifically comprises detecting an angle point of the counter frame image and classifying the counter frame image as readable or unreadable by using the multi-task network; further, the classification of the counter frame image as readable or unreadable is specifically achieved by predicting an array composed of two floating-point numbers by using a softmax activation function, so as to represent the relative probability of readability or unreadability, and the sum of the two floating-point numbers is 1; when the probability of unreadability exceeds a preset threshold, the counter frame image is classified as unreadable; In the above sub-step S34, when two characters are identified in a same character wheel frame of the counter frame image, that is, there is one character above and below in the same character wheel frame, a dynamic optimization strategy is adopted to adjust the reading of the identified character: at this time, the area of the two characters is calculated respectively, and when the area proportion of one of the characters in the character wheel frame exceeds a preset threshold, the character in the character wheel frame is the reading of the character; when the area proportions of the two characters in the character wheel frame do not exceed the preset threshold, the reading of the character in the current character wheel frame is further adjusted according to the reading of the character in the character wheel frame of the next range: if the reading of the character in the character wheel frame of the next range is 9, the reading of the character in the current character wheel frame is the upper character of the two characters; if the reading of the character in the character wheel frame of the next range is 0, the reading of the character in the current character wheel frame is the lower character of the two characters; The first water meter reading identification system is provided with a counter frame detection model, and the sub-step S31 comprises detecting a counter frame of the digital water meter image by using the counter frame detection model. The counter frame detection model adopts an improved YOLOv7 model, which uses E-ELAN, connection-based model scaling, and reparameterization, adopts Mosaic and mixed data enhancement techniques in the preprocessing part of the input, and uniformly scales the image to 640x640 size to input the backbone network; the backbone network is composed of three main components of CBS, E-ELAN and MP1, and the CBAM attention mechanism is introduced in the backbone network and the head network; the regression prediction adopts IoU Loss, and the classification prediction adopts BCE Loss to train the network and generate the corresponding weight file, wherein IoU is the intersection over union of the predicted frame and the real frame, and the BCE Loss loss function is as follows: BCE Loss = -(y_true * log(y_pred) + (1-y_true) * log(1-y_pred)) In the above formula, y_true represents the true label, y_pred represents the output predicted by the model, log represents the natural logarithm function, and * represents element-wise multiplication between elements; When the water meter image is a pointer type water meter image, the step S3 corresponds to: using the second water meter reading recognition system to perform dial plate segmentation positioning on the pointer type water meter image to obtain the positions of the centers of each sub-dial plate; determining the order of each sub-dial plate according to the positions of the centers of each sub-dial plate; obtaining the pointer reading of each sub-dial plate in order; obtaining the pointer reading of each sub-dial plate in order by neural network classification, further, performing post-processing on the dial plate reading according to the prior information obtained before; and adjusting the reading according to the prior information of the front and rear dial plate ranges.

2. The water meter reading recognition method of claim 1, wherein: The counter frame image includes four corner points: the upper left corner, the lower left corner, the upper right corner and the lower right corner, and the sub-step S33 specifically corrects the counter frame image by transmission transformation: the corner point position includes the x and y coordinates corresponding to the corner point, the four corner points of the counter frame image before correction are called p1...p4, and the four corner points after correction are called p'1...p'4, p'1 and p'2 are located at the corner positions of the upper left corner and the upper right corner, and the coordinates of p'3 are determined by the following formulas (1) and (2): x'3 = x'1 (1) In the above formula (2), [p m ,p n ] dist is the Euclidean distance between p m ,p n , and Wrct represents the width of the corrected counter graph image, i.e. the distance between p'1 and p'2 on the x-axis; p'4 is further determined by x'4 = x'2 and y'4 = y'3.

3. The water meter reading recognition method of claim 1, wherein: The first water meter reading recognition system is provided with a character recognition model, and the sub-step S34 is to use the character recognition model to perform character recognition on the counter frame image.

4. The water meter reading recognition method of claim 3, wherein: The character recognition model adopts an improved YOLOv7 model.

Images