Mechanical water meter automatic meter reading method based on deep learning OCR model

Abstract

The invention provides a mechanical water meter automatic meter reading method based on a deep learning OCR model, and belongs to the technical field of water meter automatic meter reading, and the method comprises the following steps: collecting and marking mechanical water meter pictures, and forming a data set; a mechanical water meter character detection model, a mechanical water meter character direction judgment model and a mechanical water meter character recognition model are constructed, and the data set is used for training; obtaining a mechanical water meter picture needing to be read, sequentially inputting the mechanical water meter character detection model, the water meter character direction judgment model and the mechanical water meter character recognition model, and finally obtaining a meter reading number by determining a meter reading area position and a character direction. According to the method, automatic meter reading of the mechanical water meter is realized, the positioning precision of a meter reading area is improved, the processing speed is high, the problem of large identification error caused by diversity of shooting angles of compatible water meter pictures by identifying character directions is solved, a half-character mapping rule of a reading label is introduced, the interference of half characters on the identification precision of full characters is avoided, and the character identification precision is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of automatic meter reading of water meters, and in particular relates to an automatic meter reading method for mechanical water meters based on a deep learning OCR model. Background Art

[0002] At present, some enterprises and residential areas still use the traditional manual water meter reading method. This method relies on meter readers to read one by one, which increases the operating costs of water conservancy enterprises and occupies a large amount of human resources. In addition, meter readers need to check and record data from house to house, which is a cumbersome and time-consuming process, and the meter reading efficiency is extremely low. In addition, during the manual meter reading process, due to visual fatigue, insufficient light, improper reading angles, etc., it is easy to cause inaccurate readings, which in turn affects subsequent billing and management. In addition, manual meter reading needs to be carried out according to a certain period. This regular meter reading model is difficult to achieve real-time data updates, and responds slowly to scenarios such as sudden water use or water leakage monitoring.

[0003] A new generation of smart meter reading systems has emerged, which has improved the efficiency of water meter reading to a certain extent. However, the smart meter reading systems on the market currently mainly rely on specific water meter models or brands. They have poor compatibility and adaptability for mechanical water meters of different manufacturers and models, making them difficult to be widely used. In addition, the smart meter reading systems on the market only use simple image processing methods or traditional OCR technology. When faced with interference factors such as shadows, reflections, and stains in water meter images, the recognition accuracy is often affected. In addition, some smart meter reading systems require the installation of additional sensors or the modification of the water meter itself, which increases the deployment cost and maintenance difficulty of the meter reading system.

[0004] This is a shortcoming of the prior art. Therefore, in view of the above defects in the prior art, it is very necessary to provide a mechanical water meter automatic meter reading method based on a deep learning OCR model. Summary of the invention

[0005] Aiming at the defects that the manual meter reading method in the prior art is inefficient and the readings are inaccurate, and the existing intelligent meter reading system requires a specific water meter, has poor compatibility, and increases the cost of meter reading, the present invention provides an automatic meter reading method for a mechanical water meter based on a deep learning OCR model to solve the above technical problems.

[0006] The present invention provides a method for automatic meter reading of a mechanical water meter based on a deep learning OCR model, comprising the following steps: S1. Collect and annotate mechanical water meter images to form a data set; S2. Build a mechanical water meter text detection model, a mechanical water meter text direction judgment model, and a mechanical water meter text recognition model, and use the data set for training; S3. Obtain an image of the mechanical water meter that needs to be read, and input the mechanical water meter text detection model, the water meter text direction judgment model and the mechanical water meter text recognition model in sequence, and finally obtain the meter reading by determining the meter reading area position and text direction.

[0007] Furthermore, the specific steps of step S1 are as follows: S11. Collect a set number of mechanical water meter images, annotate the text belonging to the meter reading position in the mechanical water meter images to obtain a mechanical water meter text detection data set, and divide it into a training set, a validation set, and a test set; S12. Determine the meter reading area image from the mechanical water meter image according to the meter reading location, annotate the meter reading area image with text direction labels to obtain a mechanical water meter text direction judgment data set, and divide it into a training set, a validation set, and a test set; S13. The meter reading area images are annotated using the water meter reading rules, and the semi-characters between adjacent characters are annotated with new categories to obtain the mechanical water meter text recognition dataset, which is divided into training set, validation set and test set.

[0008] Furthermore, the specific steps of step S11 are as follows: S111. Use LabelMe annotation tool to annotate the meter reading location in polygonal form on the collected mechanical water meter image to obtain an annotation file in json format; S112. Convert the annotation file in json format to txt format, where each line of data represents a meter reading location in a mechanical water meter image, and each line of data is the coordinates of each vertex of a polygon; S113. Use the imgaug image tool to perform data enhancement on the mechanical water meter image and the coordinate information of the meter reading location to expand the data volume and obtain a mechanical water meter text detection data set; S114. Divide the mechanical water meter text detection dataset into a training set, a validation set, and a test set according to a set ratio.

[0009] Furthermore, the specific steps of step S12 are as follows: S121. Determine the meter reading area image from the mechanical water meter image according to the coordinate information of the meter reading location; S122. Setting text direction labels for the meter reading area image at 0° and 180°, wherein the forward direction is set to a text direction label of 0°, and the reverse direction is set to a text direction label of 180°; S123. The meter reading area images and text direction labels are used to form a mechanical water meter text direction judgment data set, and divided into a training set, a validation set, and a test set according to a set ratio.

[0010] Furthermore, the specific steps of step S13 are as follows: S131. After rotating the meter reading area image with the text direction label as the direction, mark the readings according to the meter reading rules and the actual readings of 0-9 characters to obtain the meter reading label; S132. A new category is defined for the position between adjacent characters where characters 0-9 appear at the end of the meter reading area image as a half-character for annotation, and the meter reading area image and meter reading labels are formed into a mechanical water meter text recognition data set, and divided into a training set, a validation set, and a test set according to a set ratio.

[0011] Furthermore, the specific steps of step S2 are as follows: S21. Build a mechanical water meter text detection model based on the DB network, define model training parameters and evaluation indicators, and use the training set of the mechanical water meter text detection data set to train the DB network, and use the validation set to tune the training parameters until the evaluation indicators meet the requirements, output the optimal mechanical water meter text detection model, and test it through the test set; S22. Build a mechanical water meter text direction judgment model based on the MobileNetV3 network, define the model's training parameters and evaluation indicators, and use the training set of the mechanical water meter text direction judgment data set to train the MobileNetV3 network, and use the validation set to tune the training parameters until the evaluation indicators meet the requirements, output the optimal mechanical water meter text direction judgment model, and test it through the test set; S23. Build a mechanical water meter text recognition model based on the SVTR network, define the model's training parameters and evaluation indicators, use the training set of the mechanical water meter text recognition dataset to train the SVTR network, and use the validation set to tune the training parameters until the evaluation indicators meet the requirements, output the optimal mechanical water meter text recognition model, and test it through the test set.

[0012] Furthermore, the specific steps of step S21 are as follows: S211. A DB network is built for the purpose of constructing a mechanical water meter text detection model, with a picture of a mechanical water meter that needs to be read as input and a binary segmentation map of the mechanical water meter picture as output; S212. Use the ResNet18 network with deformable convolution as the backbone network of the DB network, use the standard FPN network as the backbone network of the DB network, and use the probability prediction branch, threshold calculation branch, and approximate binarization branch as the head network of the DB network; S213. Define the classification threshold loss, probability graph segmentation loss and approximate binary graph segmentation loss of the DB network, define the model training rounds, batch data size, optimizer, initial learning rate, weight decay, momentum coefficient, learning rate update formula, maximum number of iterations, and learning rate decay coefficient of the DB network; S214. After inputting the image of the mechanical water meter that needs to be read into the DB network for data enhancement of flipping, rotation and scaling, the optimizer is initialized using the initial learning rate, and the iterative training of each training round is started according to the set batch data size. The learning rate update formula is used to update the learning rate during the training process, and after the batch processing of each training round is completed, the gradient of the current batch is used to update the model parameters of the weights and biases of the backbone network, the neck network and the head network; S215. At the end of each training round, the overall loss is calculated using the following formula :

[0013] in, represents the classification threshold loss, represents the probability map segmentation loss, represents the approximate binary image segmentation loss, is the approximate binary image segmentation loss coefficient, is the classification threshold loss coefficient; S216. After each training round, the validation set is used to calculate the precision, recall and average recall of each trained mechanical water meter text detection model; S217. Filter out the optimal mechanical water meter text detection model based on the overall loss, model precision, recall rate and average recall rate.

[0014] Furthermore, the specific steps of step S22 are as follows: S221. A MobileNetV3 network is built for the purpose of constructing a mechanical water meter file direction judgment model, with the meter reading picture that needs to be read as input and the text direction angle of the meter reading area picture as output; S222. Stack several block channels based on separable convolution, SE channel attention mechanism and residual link to form a backbone network; S223. Define the training rounds, batch data size, optimizer, initial learning rate, and learning rate update formula of the MobileNetV3 network; S224. Input the meter reading area image that needs to be read into the MobileNetV3 network, use the initialization learning rate to initialize the optimizer, start the iterative training of each training round according to the set batch data, and use the learning rate update formula to update the learning rate during the training process, and after the batch processing of each training round is completed, use the gradient update of the current batch as the model parameters of the weight and bias of the backbone network; S225. At the end of each training round, the validation set is used to calculate the recognition accuracy of each trained mechanical water meter text direction judgment model, and the optimal mechanical water meter text direction judgment model is selected based on the recognition accuracy.

[0015] Furthermore, the specific steps of step S23 are as follows: S231. Build an SVTR network for the purpose of building a mechanical water meter text recognition model, take a meter reading picture with a meter reading direction of 0º as input, and output the decoded sequence of the meter reading as the meter reading; S232. The MobileV1 network with enhanced strategy is used as the backbone network of the SVTR network, and the CTC network and SAR network are used as the head network of the SVTR network; S233 defines the training rounds, batch data size, optimizer, initial learning rate, and learning rate update formula of the SVTR network; S234. After scaling the meter reading area image to a uniform size, use the initialization learning rate to initialize the optimizer, start the iterative training of each training round according to the set batch data size, and use the learning rate update formula to update the learning rate during the training process, and after the batch processing of each training round is completed, use the gradient update of the current batch as the model parameters of the weights and biases of the backbone network and the head network; S235. Calculate the sum of CTC loss and SAR loss at the end of each training round as the overall loss; S236. After each training round, the error and accuracy of each trained mechanical water meter text recognition model are calculated using the validation set; S237. Filter out the optimal mechanical water meter text recognition model based on the overall loss, model error and accuracy.

[0016] Furthermore, the specific steps of step S3 are as follows: S31. Obtain the image of the mechanical water meter that needs to be read, input the coordinate information of each vertex of the polygon representing the meter reading position into the tested mechanical water meter text detection model, and calculate the coordinate information of the rotated rectangle of the meter reading area through the minimum circumscribed rectangle algorithm of the OpenCV visual library, and use the perspective transformation of the OpenCV visual library to obtain the final meter reading area image; S32. Input the meter reading area image that needs to be read into the tested mechanical water meter text direction judgment model to obtain the text direction of the meter reading area image; When the text direction is positive, go to step S34; When the text direction is reverse, go to step S33; S33. Rotate the meter reading area image that needs to be read by 180 degrees; S34. Input the meter reading area image into the tested mechanical water meter text recognition model to obtain the characterization meter reading, and determine whether there is a half-character label in the meter reading logarithm; If yes, go to step S36; If not, proceed to step S35; S35. Query the corresponding numbers before and after the half-character label, and replace the half-character with the small character in front of the half-character label; S36. Output the meter reading.

[0017] The beneficial effects of the present invention are: The automatic meter reading method for mechanical water meters based on the deep learning OCR model provided by the present invention realizes automatic meter reading of mechanical water meters, performs text detection based on the segmentation results of the DB network, and can be applied to water meter pictures taken at various shooting angles, thereby improving the accuracy of positioning the meter reading area. ResNet18 is used as the backbone of the DB network, and the processing speed is faster. The present invention uses a classification network to judge the direction of the text, which solves the problem of large recognition errors caused by the diversification of shooting angles of water meter pictures. The SVTR text recognition network of the present invention is based on a single visual model, and has a faster reasoning speed and higher text recognition accuracy than the mainstream algorithm. The present invention introduces a half-character mapping rule for the reading label, which avoids the interference of half-characters on the recognition accuracy of full characters, and greatly improves the text recognition accuracy. The models of the present invention have strong generalization capabilities, and can be applied to complex scenes, various types of mechanical water meters, and unified reading recognition.

[0018] In addition, the invention has a reliable design principle, a simple structure and a very broad application prospect.

[0019] It can be seen that compared with the prior art, the present invention has outstanding substantive features and significant progress, and the beneficial effects of its implementation are also obvious. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0021] Figure 1 It is a flow chart of the automatic meter reading method of a mechanical water meter based on a deep learning OCR model of the present invention.

[0022] Figure 2 It is a network structure diagram of the mechanical water meter text detection model of the present invention.

[0023] Figure 3 yes Figure 2 A magnified diagram of the input end image.

[0024] Figure 4 yes Figure 2 A magnified schematic diagram of the output end image.

[0025] Figure 5 It is a schematic diagram of the learning rate change in the training of the mechanical water meter text detection model of the present invention.

[0026] Figure 6 It is a schematic diagram of the changes in various types of errors in the training of the mechanical water meter text detection model of the present invention and the accuracy and recall rate of the verification set.

[0027] Figure 7 It is a schematic diagram of the detection results of the mechanical water meter text detection model of the present invention.

[0028] Figure 8 It is a network structure diagram of the text direction judgment model of the mechanical water meter of the present invention.

[0029] Fig. 9 It is a schematic diagram of the learning rate change in the training of the mechanical water meter text direction judgment model of the present invention.

[0030] Fig.10 It is a schematic diagram of classification loss and classification accuracy in the training of the mechanical water meter text direction judgment model of the present invention.

[0031] Fig.11 It is a schematic diagram of the detection results of the text direction judgment model of the mechanical water meter of the present invention.

[0032] Fig.12 It is a network structure diagram of the mechanical water meter text recognition model of the present invention.

[0033] Fig.13 It is a schematic diagram of the learning rate change in the training of the mechanical water meter text recognition model of the present invention.

[0034] Fig.14 It is a schematic diagram of the changes in various types of errors and the accuracy of the verification set in the training of the mechanical water meter text recognition model of the present invention.

[0035] Fig.15 It is a schematic diagram of the detection results of the mechanical water meter text recognition model of the present invention. DETAILED DESCRIPTION

[0036] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0037] See also Figure 1 The figure shows a specific implementation of a mechanical water meter automatic meter reading method based on a deep learning OCR model, including the following steps: S1. Collect and annotate mechanical water meter images to form a data set; S2. Build a mechanical water meter text detection model, a mechanical water meter text direction judgment model, and a mechanical water meter text recognition model, and use the data set for training; S3. Obtain an image of the mechanical water meter that needs to be read, and input the mechanical water meter text detection model, the water meter text direction judgment model and the mechanical water meter text recognition model in sequence, and finally obtain the meter reading by determining the meter reading area position and text direction.

[0038] In this embodiment, the specific steps of step S1 are as follows: S11. Collect a set number of mechanical water meter images, annotate the text belonging to the meter reading position in the mechanical water meter images to obtain a mechanical water meter text detection data set, and divide it into a training set, a validation set and a test set; the specific steps of step S11 are as follows: S111. Use LabelMe annotation tool to annotate the meter reading location in polygonal form on the collected mechanical water meter image to obtain an annotation file in json format; S112. Convert the annotation file in json format to txt format, where each line of data represents the meter reading location in a mechanical water meter image, and each line of data is the coordinates of each vertex of the polygon; for example, the annotation file in txt format of the UI of each mechanical water meter image should be (x1, y1, x2, y2, x3, y3, x4, y4), which respectively represent the coordinates of four points in the meter reading area; S113. Use the imgaug image tool to perform data enhancement on the mechanical water meter image and the coordinate information of the meter reading location to expand the data volume and obtain a mechanical water meter text detection data set; S114. Divide the mechanical water meter text detection data set into a training set, a validation set, and a test set according to a set ratio; for example, the training set ratio is 70%, the validation set ratio is 20%, and the test set ratio is 10%; S12. Determine the meter reading area image from the mechanical water meter image according to the meter reading location, annotate the meter reading area image with text direction labels to obtain a mechanical water meter text direction judgment data set, and divide it into a training set, a validation set, and a test set; the specific steps of step S12 are as follows: S121. Determine the meter reading area image from the mechanical water meter image according to the coordinate information of the meter reading location; S122. Setting text direction labels for the meter reading area image at 0° and 180°, wherein the forward direction is set to a text direction label of 0°, and the reverse direction is set to a text direction label of 180°; S123. The meter reading area images and text direction labels are used to form a mechanical water meter text direction judgment data set, and divided into a training set, a validation set, and a test set according to a set ratio; S13. The meter reading area image is annotated using the water meter reading rules, and the semi-characters between adjacent characters are annotated with new categories to obtain a mechanical water meter text recognition data set, which is divided into a training set, a validation set, and a test set; the specific steps of step S13 are as follows: S131. After rotating the meter reading area image with the text direction label as the direction, mark the readings according to the meter reading rules and the actual readings of 0-9 characters to obtain the meter reading label; S132. Define a new category as a half-character for the position between adjacent characters where characters 0-9 will appear at the end of the meter reading area image, and form a mechanical water meter text recognition data set with meter reading area images and meter reading labels, and divide it into training set, validation set and test set according to the set ratio; It should be noted that when a certain digit of the water meter reading is between two digits, that is, in the case of a "half character", this article marks it as a new category, for example, the "A", "B"-"J" categories can be used to represent "01", "12"-"90" half characters; The specific steps of step S2 are as follows: S21. Build a mechanical water meter text detection model based on the DB network, define model training parameters and evaluation indicators, and use the training set of the mechanical water meter text detection data set to train the DB network, and use the validation set to tune the training parameters until the evaluation indicators meet the requirements, output the optimal mechanical water meter text detection model, and test it with the test set; the specific steps of step S21 are as follows: S211. A DB network is built for the purpose of constructing a mechanical water meter text detection model, with a picture of a mechanical water meter that needs to be read as input and a binary segmentation map of the mechanical water meter picture as output; S212. Use the ResNet18 network with deformable convolution as the backbone network of the DB network, use the standard FPN network as the backbone network of the DB network, and use the probability prediction branch, threshold calculation branch, and approximate binarization branch as the head network of the DB network; S213. Define the classification threshold loss, probability graph segmentation loss and approximate binary graph segmentation loss of the DB network, define the model training rounds, batch data size, optimizer, initial learning rate, weight decay, momentum coefficient, learning rate update formula, maximum number of iterations, and learning rate decay coefficient of the DB network; for example, the training rounds are 1200 times, and the batch data size is 8 mechanical water meter pictures; the optimizer selects the SGD optimizer with dynamically updated learning rate, the initial learning rate is 0.007, the weight decay is 0.0001, and the momentum coefficient is 0.9; the learning rate update formula adopts ,in is the decay exponent of the learning rate, Indicates the current number of iterations, ranging from [1, ], is the maximum number of iterations, which can be 105600; S214. After inputting the image of the mechanical water meter that needs to be read into the DB network for data enhancement of flipping, rotation and scaling, the optimizer is initialized using the initial learning rate, and the iterative training of each training round is started according to the set batch data size. The learning rate update formula is used to update the learning rate during the training process, and after the batch processing of each training round is completed, the gradient of the current batch is used to update the model parameters of the weights and biases of the backbone network, the neck network and the head network; S215. At the end of each training round, the overall loss is calculated using the following formula :

[0039] in, represents the classification threshold loss, represents the probability map segmentation loss, Represents the approximate binary image segmentation loss; is the approximate binary image segmentation loss coefficient, for example, it can be 5. is the classification threshold loss coefficient, for example, it can be 10; S216. After each training round, the validation set is used to calculate the precision, recall and average recall of each trained mechanical water meter text detection model; S217. Filter out the best mechanical water meter text detection model according to the overall loss, the precision rate, the recall rate and the average recall rate of the model; for example, the precision rate, the recall rate and the average recall rate of the model are greater than a certain set threshold, and the overall loss needs to be reduced to a loss threshold that is considered stable to filter out the mechanical water meter text detection model; like Figure 2 In the network structure diagram of the mechanical water meter text detection model shown in FIG, each square box represents a feature map. represents feature map addition, conv represents 3*3 convolution, up×n represents n-fold upsampling, CON-CAT represents feature map concatenation, and DB represents differentiable binarization. Figure 3 yes Figure 2 A magnified schematic diagram of the input end image. Figure 4 yes Figure 2 The enlarged diagram of the output image in the figure. The green box is the meter reading area in the txt format. Figure 5 This is a diagram showing the change in learning rate during the training of the mechanical water meter text detection model. Figure 6 (a) shows the change of overall error during training. As the number of training iterations increases, the error becomes smaller and smaller, that is, the model detection ability becomes stronger and stronger; Figure 6 (b) shows the change of classification threshold error during training. As the number of training iterations increases, the error becomes smaller and smaller, that is, the confidence becomes higher and higher; Figure 6 (c) shows the change of probability map segmentation error during training. As the number of training iterations increases, the error becomes smaller and smaller, that is, the segmentation accuracy becomes higher and higher; Figure 6 (d) shows the change of binary image segmentation error during training. As the number of training iterations increases, the error becomes smaller and smaller, that is, the accuracy of binary segmentation becomes higher and higher; Figure 6 (e) represents the accuracy during the verification process. The value has always remained close to 1, indicating that the model can accurately locate the water meter area to be detected. Since the pre-trained model itself is for general scene text detection, the model accuracy has always remained close to 1. Figure 6 (f) represents the recall rate during the verification process. As the number of training iterations increases, the recall rate increases, indicating that the model's missed detection rate decreases; Figure 6 (g) Average precision and recall during the validation process. As the number of training iterations increases, the average precision and recall increases, indicating that the model performance tends to be robust. S22. Build a mechanical water meter text direction judgment model based on the MobileNetV3 network, define the model's training parameters and evaluation indicators, and use the training set of the mechanical water meter text direction judgment data set to train the MobileNetV3 network, and use the validation set to tune the training parameters until the evaluation indicators meet the requirements, output the optimal mechanical water meter text direction judgment model, and test it through the test set; the specific steps of step S22 are as follows: S221. A MobileNetV3 network is built for the purpose of constructing a mechanical water meter document direction judgment model, with the meter reading picture that needs to be read as input and the text direction angle of the meter reading area picture as output; S222. Stack several block channels based on separable convolution, SE channel attention mechanism and residual link to form a backbone network; MobileNetV3 network structure is as follows Figure 8 As shown; S223. Define the training rounds, batch data size, optimizer, initial learning rate, and learning rate update formula of the MobileNetV3 network; for example, the training rounds is 100, the batch data size is 16 meter reading area pictures, the optimizer uses the Adam optimizer with a cosine return strategy to update the learning rate, the initial learning rate is 0.0001, and the learning rate update formula is as follows: ,in Indicates the current training round, ranging from [1, ]; the learning rate is dynamically updated during training, such as Fig. 9 As shown; S224. After scaling the meter reading area image to be read to a uniform size of 192 x 48, input it into the MobileNetV3 network, initialize the optimizer using the initial learning rate, start iterative training of each training round according to the set batch data, and use the learning rate update formula to update the learning rate during the training process. After the batch processing of each training round is completed, use the gradient update of the current batch as the model parameters of the weight and bias of the backbone network; S225. At the end of each training round, the validation set is used to calculate the recognition accuracy of each trained mechanical water meter text direction judgment model, and the optimal mechanical water meter text direction judgment model is selected according to the recognition accuracy; for example, when the recognition accuracy reaches the set threshold and the classification loss drops to the basically stable set loss threshold, the mechanical water meter text direction judgment model is selected; the optimal text direction judgment model training process is as follows Fig.10 As shown in the figure, some direction judgment effect diagrams are as follows Fig.11 As shown; Fig.10(a) shows the change of classification error during training. As the number of training iterations increases, the error becomes smaller and smaller, that is, the classification ability of the model becomes stronger and stronger; Fig.10 (b) represents the accuracy during the verification process. The higher the accuracy, the better the classification model performance. S23. Build a mechanical water meter text recognition model based on the SVTR network, define the training parameters and evaluation indicators of the model, and use the training set of the mechanical water meter text recognition data set to train the SVTR network, and use the validation set to tune the training parameters until the evaluation indicators meet the requirements, output the optimal mechanical water meter text recognition model, and test it through the test set; the specific steps of step S23 are as follows: S231. Build an SVTR network for the purpose of building a mechanical water meter text recognition model, take a meter reading picture with a meter reading direction of 0° as input, and output the decoded sequence of the meter reading as the meter reading; S232. The MobileV1 network with enhanced strategy is used as the backbone network of the SVTR network, and the CTC network and SAR network are used as the head network of the SVTR network; the SVTR network structure is as follows Fig.12 As shown; S233. Define the training rounds, batch data size, optimizer, initial learning rate, and learning rate update formula of the SVTR network; for example, the training rounds The number of updates is 1000, the batch size is 64, the initial learning rate is 0.001, and the Adam optimizer with warm-up strategy and step-by-step strategy is used to update the learning rate. The update formula of the warm-up strategy learning rate is the following step-by-step strategy learning rate update formula: ; During the training process, the learning rate is dynamically updated as follows Fig.13 As shown; S234. After scaling the meter reading area image to be read to a uniform size of 320 in width and 48 in height, use the initial learning rate to initialize the optimizer, start iterative training of each training round according to the set batch data size, and use the learning rate update formula to update the learning rate during the training process, and after the batch processing of each training round is completed, use the gradient update of the current batch as the model parameters of the weights and biases of the backbone network and the head network; S235. Calculate the sum of CTC loss and SAR loss at the end of each training round as the overall loss; S236. After each training round, the error and accuracy of each trained mechanical water meter text recognition model are calculated using the validation set; S237. Filter out the best mechanical water meter text recognition model based on the overall loss, model error and accuracy; for example, the mechanical water meter text recognition model is selected when the recognition accuracy reaches the set threshold, the error is less than the set threshold, and the classification loss drops to the set loss threshold that is basically stable; The optimal text recognition model training process is as follows Fig.14 As shown in Figure 2, some text recognition results are as follows: Fig.15 As shown; Fig.14 (a) shows the change of overall error during training. As the number of training iterations increases, the error becomes smaller and smaller, that is, the model recognition ability becomes stronger and stronger; Fig.14 (b) shows the change of CTC error during training. As the number of training iterations increases, the error becomes smaller and smaller, that is, higher and higher; Fig.14 (c) shows the change of SAR error during the training process. As the number of training iterations increases, the error becomes smaller and smaller, that is, it becomes higher and higher; Fig.14 (d) represents the accuracy during the verification process. As the number of training iterations increases, the accuracy increases, indicating that the model's recognition ability becomes stronger and stronger; Fig.14 (e) represents the normalized edit distance during the verification process. As the number of training iterations increases, the edit distance becomes larger and larger, indicating that the model recognition ability is getting stronger and stronger; The specific steps of step S3 are as follows: S31. Obtain the image of the mechanical water meter that needs to be read, input the coordinate information of each vertex of the polygon representing the meter reading position into the tested mechanical water meter text detection model, and calculate the coordinate information of the rotated rectangle of the meter reading area through the minimum circumscribed rectangle algorithm of the OpenCV visual library, and use the perspective transformation of the OpenCV visual library to obtain the final meter reading area image; For example, the meter reading area coordinate information output by the DB model of the mechanical water meter text detection model is an irregular box, such as (x1, y1, ..., xn, yn), where n>=4, which needs to be converted into a rectangular box (x1, y1, x2, y2, x3, y3, x4, y4) for subsequent use; the specific conversion process is as follows: first, use the minimum bounding rectangle method of OpenCV to obtain the rotated rectangular coordinate information of the meter reading area; then use the perspective transformation of OpenCV to obtain the meter reading area image from the mechanical water meter image; Figure 7 As shown, it is necessary to Figure 7 The meter reading area shown in (a) is converted into a rectangular frame. The specific conversion process is as follows: First, the meter reading area coordinates are transformed, and the minimum enclosing rectangle method of OpenCV is used to obtain the following: Figure 7 (b) shows the coordinate information of the rotated rectangle of the meter reading area; then, the meter reading area image is obtained, and the perspective transformation of OpenCV is used to obtain the following from the mechanical water meter image: Figure 7(c) Picture of the meter reading area shown; S32. Input the meter reading area image that needs to be read into the tested mechanical water meter text direction judgment model to obtain the text direction of the meter reading area image; When the text direction is positive, go to step S34; When the text direction is reverse, go to step S33; S33. Rotate the meter reading area image that needs to be read by 180 degrees; S34. Input the meter reading area image into the tested mechanical water meter text recognition model to obtain the characterization meter reading, and determine whether there is a half-character label in the meter reading logarithm; If yes, go to step S36; If not, proceed to step S35; S35. Query the corresponding numbers before and after the half-character label, and replace the half-character with the small character in front of the half-character label; S36. Output the meter reading.

[0040] It should be understood that the order of execution of the steps in the above embodiment does not mean that the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0041] Although the present invention has been described in detail with reference to the accompanying drawings and in combination with the preferred embodiments, the present invention is not limited thereto. Without departing from the spirit and essence of the present invention, a person of ordinary skill in the art may make various equivalent modifications or substitutions to the embodiments of the present invention, and these modifications or substitutions shall be within the scope of the present invention. Any person of ordinary skill in the art may easily think of changes or substitutions within the technical scope disclosed by the present invention, and these shall be within the scope of protection of the present invention.

Claims

1. A mechanical water meter automatic meter reading method based on deep learning OCR model, characterized in that: The steps include: S1. Collect and annotate mechanical water meter images to form a data set; S2. Build a mechanical water meter text detection model, a mechanical water meter text direction judgment model, and a mechanical water meter text recognition model, and use the data set for training; S3. Obtain an image of the mechanical water meter that needs to be read, and input the mechanical water meter text detection model, the water meter text direction judgment model and the mechanical water meter text recognition model in sequence, and finally obtain the meter reading by determining the meter reading area position and text direction.

2. The automatic meter reading method for mechanical water meters based on deep learning OCR model according to claim 1, characterized in that: The specific steps of step S1 are as follows: S11. Collect a set number of mechanical water meter images, annotate the text belonging to the meter reading position in the mechanical water meter images to obtain a mechanical water meter text detection data set, and divide it into a training set, a validation set, and a test set; S12. Determine the meter reading area image from the mechanical water meter image according to the meter reading location, annotate the meter reading area image with text direction labels to obtain a mechanical water meter text direction judgment data set, and divide it into a training set, a validation set, and a test set; S13. The meter reading area images are annotated using the water meter reading rules, and the semi-characters between adjacent characters are annotated with new categories to obtain the mechanical water meter text recognition dataset, which is divided into training set, validation set and test set.

3. The automatic meter reading method for mechanical water meters based on deep learning OCR model as claimed in claim 2, characterized in that: The specific steps of step S11 are as follows: S111. Use LabelMe annotation tool to annotate the meter reading location in polygonal form on the collected mechanical water meter image to obtain an annotation file in json format; S112. Convert the annotation file in json format to txt format, where each line of data represents a meter reading location in a mechanical water meter image, and each line of data is the coordinates of each vertex of a polygon; S113. Use the imgaug image tool to perform data enhancement on the mechanical water meter image and the coordinate information of the meter reading location to expand the data volume and obtain a mechanical water meter text detection data set; S114. Divide the mechanical water meter text detection dataset into a training set, a validation set, and a test set according to a set ratio.

4. The automatic meter reading method for mechanical water meters based on deep learning OCR model as claimed in claim 2, characterized in that: The specific steps of step S12 are as follows: S121. Determine the meter reading area image from the mechanical water meter image according to the coordinate information of the meter reading location; S122. Setting text direction labels for the meter reading area image at 0° and 180°, wherein the forward direction is set to a text direction label of 0°, and the reverse direction is set to a text direction label of 180°; S123. The meter reading area images and text direction labels are used to form a mechanical water meter text direction judgment data set, and divided into a training set, a validation set, and a test set according to a set ratio.

5. The automatic meter reading method for mechanical water meters based on deep learning OCR model as claimed in claim 3, characterized in that: The specific steps of step S13 are as follows: S131. After rotating the meter reading area image with the text direction label as the direction, mark the readings according to the meter reading rules and the actual readings of 0-9 characters to obtain the meter reading label; S132. A new category is defined for the position between adjacent characters where characters 0-9 appear at the end of the meter reading area image as a half-character for annotation, and the meter reading area image and meter reading labels are formed into a mechanical water meter text recognition data set, and divided into a training set, a validation set, and a test set according to a set ratio.

6. The automatic meter reading method for mechanical water meters based on deep learning OCR model as claimed in claim 3, characterized in that: The specific steps of step S2 are as follows: S21. Build a mechanical water meter text detection model based on the DB network, define model training parameters and evaluation indicators, and use the training set of the mechanical water meter text detection data set to train the DB network, and use the validation set to tune the training parameters until the evaluation indicators meet the requirements, output the optimal mechanical water meter text detection model, and test it through the test set; S22. Build a mechanical water meter text direction judgment model based on the MobileNetV3 network, define the model's training parameters and evaluation indicators, and use the training set of the mechanical water meter text direction judgment data set to train the MobileNetV3 network, and use the validation set to tune the training parameters until the evaluation indicators meet the requirements, output the optimal mechanical water meter text direction judgment model, and test it through the test set; S23. Build a mechanical water meter text recognition model based on the SVTR network, define the model's training parameters and evaluation indicators, use the training set of the mechanical water meter text recognition dataset to train the SVTR network, and use the validation set to tune the training parameters until the evaluation indicators meet the requirements, output the optimal mechanical water meter text recognition model, and test it through the test set.

7. The automatic meter reading method for mechanical water meters based on deep learning OCR model as claimed in claim 6, characterized in that: The specific steps of step S21 are as follows: S211. A DB network is built for the purpose of constructing a mechanical water meter text detection model, with a picture of a mechanical water meter that needs to be read as input and a binary segmentation map of the mechanical water meter picture as output; S212. Use the ResNet18 network with deformable convolution as the backbone network of the DB network, use the standard FPN network as the backbone network of the DB network, and use the probability prediction branch, threshold calculation branch, and approximate binarization branch as the head network of the DB network; S213. Define the classification threshold loss, probability graph segmentation loss and approximate binary graph segmentation loss of the DB network, define the model training rounds, batch data size, optimizer, initial learning rate, weight decay, momentum coefficient, learning rate update formula, maximum number of iterations, and learning rate decay coefficient of the DB network; S214. After inputting the image of the mechanical water meter that needs to be read into the DB network for data enhancement of flipping, rotation and scaling, the optimizer is initialized using the initial learning rate, and the iterative training of each training round is started according to the set batch data size. The learning rate update formula is used to update the learning rate during the training process, and after the batch processing of each training round is completed, the gradient of the current batch is used to update the model parameters of the weights and biases of the backbone network, the neck network and the head network; S215. At the end of each training round, the overall loss is calculated using the following formula : in, represents the classification threshold loss, represents the probability map segmentation loss, represents the approximate binary image segmentation loss, is the approximate binary image segmentation loss coefficient, is the classification threshold loss coefficient; S216. After each training round, the validation set is used to calculate the precision, recall and average recall of each trained mechanical water meter text detection model; S217. Filter out the optimal mechanical water meter text detection model based on the overall loss, model precision, recall rate and average recall rate.

8. The automatic meter reading method for mechanical water meters based on deep learning OCR model as claimed in claim 6, characterized in that: The specific steps of step S22 are as follows: S221. A MobileNetV3 network is built for the purpose of constructing a mechanical water meter document direction judgment model, with the meter reading picture that needs to be read as input and the text direction angle of the meter reading area picture as output; S222. Stack several block channels based on separable convolution, SE channel attention mechanism and residual link to form a backbone network; S223. Define the training rounds, batch data size, optimizer, initial learning rate, and learning rate update formula of the MobileNetV3 network; S224. Input the meter reading area image that needs to be read into the MobileNetV3 network, use the initialization learning rate to initialize the optimizer, start the iterative training of each training round according to the set batch data, and use the learning rate update formula to update the learning rate during the training process, and after the batch processing of each training round is completed, use the gradient update of the current batch as the model parameters of the weight and bias of the backbone network; S225. At the end of each training round, the validation set is used to calculate the recognition accuracy of each trained mechanical water meter text direction judgment model, and the optimal mechanical water meter text direction judgment model is selected based on the recognition accuracy.

9. The automatic meter reading method for mechanical water meters based on deep learning OCR model as claimed in claim 6, characterized in that: The specific steps of step S23 are as follows: S231. Build an SVTR network for the purpose of building a mechanical water meter text recognition model, take a meter reading picture with a meter reading direction of 0º as input, and output the decoded sequence of the meter reading as the meter reading; S232. The MobileV1 network with enhanced strategy is used as the backbone network of the SVTR network, and the CTC network and SAR network are used as the head network of the SVTR network; S233 defines the training rounds, batch data size, optimizer, initial learning rate, and learning rate update formula of the SVTR network; S234. After scaling the meter reading area image to a uniform size, use the initialization learning rate to initialize the optimizer, start the iterative training of each training round according to the set batch data size, and use the learning rate update formula to update the learning rate during the training process, and after the batch processing of each training round is completed, use the gradient update of the current batch as the model parameters of the weights and biases of the backbone network and the head network; S235. Calculate the sum of CTC loss and SAR loss at the end of each training round as the overall loss; S236. After each training round, the error and accuracy of each trained mechanical water meter text recognition model are calculated using the validation set; S237. Filter out the optimal mechanical water meter text recognition model based on the overall loss, model error and accuracy.

10. The automatic meter reading method for mechanical water meters based on deep learning OCR model as claimed in claim 4, characterized in that: The specific steps of step S3 are as follows: S31. Obtain the image of the mechanical water meter that needs to be read, input the coordinate information of each vertex of the polygon representing the meter reading position into the tested mechanical water meter text detection model, and calculate the coordinate information of the rotated rectangle of the meter reading area through the minimum circumscribed rectangle algorithm of the OpenCV visual library, and use the perspective transformation of the OpenCV visual library to obtain the final meter reading area image; S32. Input the meter reading area image that needs to be read into the tested mechanical water meter text direction judgment model to obtain the text direction of the meter reading area image; When the text direction is positive, go to step S34; When the text direction is reverse, go to step S33; S33. Rotate the meter reading area image that needs to be read by 180 degrees; S34. Input the meter reading area image into the tested mechanical water meter text recognition model to obtain the characterization meter reading, and determine whether there is a half-character label in the meter reading logarithm; If yes, go to step S36; If not, proceed to step S35; S35. Query the corresponding numbers before and after the half-character label, and replace the half-character with the small character in front of the half-character label; S36. Output the meter reading.

Images