Intelligent water meter image quality automatic evaluation method based on deep learning

By developing an automatic image quality assessment method for smart water meters based on deep learning, which utilizes convolutional neural networks and Transformer encoders for feature extraction, the accuracy and adaptability issues of water meter image quality assessment are solved, achieving efficient image quality assessment.

CN118941927APending Publication Date: 2024-11-12BASIC INTELLIGENCE TECH CO LTD

Patent Information

Application Number
CN202411121778.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-15
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

In existing technologies, subjective image quality assessment methods are inefficient and costly, traditional objective image quality assessment methods cannot adapt to the diversity of water meter image distortion, and traditional no-reference image quality assessment methods have low model accuracy and weak generalization performance.

Method used

An automatic image quality assessment method for smart water meters based on deep learning is adopted. It uses convolutional neural networks and Transformer encoders for feature extraction, and combines fully connected layers for quality assessment score prediction to construct a no-reference image quality assessment model.

Benefits of technology

It achieves accurate and real-time water meter image quality assessment, is suitable for large-scale data processing and automation systems, and can flexibly adapt to image distortion under different environments and conditions, thus improving the accuracy and robustness of image quality prediction.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118941927A_ABST
    Figure CN118941927A_ABST
Patent Text Reader

Abstract

The invention discloses an intelligent water meter image quality automatic evaluation method based on deep learning, and the method is based on a deep learning technology, designs a non-reference image quality evaluation model, does not need a fixed reference image, is more suitable for an actual application scene, and can better meet the image quality evaluation requirements under different environments and conditions. Besides, the non-reference image quality evaluation model combines a convolutional neural network and a Transform-based encoder to realize extraction of local features and non-local features of the water meter image, so that image information can be extracted more accurately, local and global features of the image can be captured better, and the accuracy of the water meter quality evaluation is improved. Therefore, the accuracy and robustness of image quality evaluation prediction are improved, automatic evaluation of the image quality of the intelligent water meter is realized, the image acquisition quality of the intelligent water meter equipment is detected, the water meter image data recognition accuracy is further improved, and better support is provided for maintenance and use of the intelligent water meter equipment.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of image quality assessment, and in particular to an automatic assessment method for smart water meter image quality based on deep learning. Background Art

[0002] With the advancement of smart city construction, the traditional water meter reading method is gradually changing to the vision-based smart water meter reading method. This new meter reading method requires taking a large number of water meter images to achieve automatic meter reading and data collection. However, due to the complexity of the water meter working environment, such as lighting conditions, shooting angles, water meter surface conditions and other factors, the captured water meter images often have various distortion problems, such as blur, noise, uneven lighting, occlusion, etc. These image distortion problems may have an adverse effect on subsequent water meter reading recognition and data analysis. Therefore, the automated evaluation of the quality of smart water meter images has become crucial. By evaluating the quality of the captured water meter images, the clarity and recognizability of the images are detected, thereby ensuring the accuracy of subsequent image reading and recognition.

[0003] Image quality assessment (evaluation) is one of the hot research issues in the field of computer vision. Depending on whether there is human participation, it can be divided into subjective image quality evaluation and objective image quality evaluation. Subjective image quality evaluation relies on human subjective feelings to judge the quality of the image; while objective image quality evaluation requires the use of computers to establish mathematical models, and then calculate and output digital measurement results, and try to require that the objective image quality evaluation results and subjective image quality evaluation scores maintain a high degree of consistency.

[0004] The above subjective image quality evaluation methods are characterized by low efficiency and high cost, and it is difficult to achieve accurate and real-time quality evaluation. Therefore, it is an inevitable trend to study objective image quality evaluation algorithms. Early objective image quality evaluation algorithms mainly targeted specific types of distortion, such as image compression distortion, block effect distortion, and image blur, etc. However, due to the diversity of image distortion in practical applications, image quality evaluation methods for non-specific distortion are more practical.

[0005] According to whether there is reference information, objective image quality assessment methods are further divided into three categories: Full-Reference Image Quality Assessment (FR-IQA), Reduced-Reference Image Quality Assessment (RR-IQA) and No-Reference Image Quality Assessment (NR-IQA).

[0006] The full-reference image quality assessment method mainly uses the difference between the distorted image and the ideal reference image to evaluate the quality of the distorted image. The semi-reference image quality assessment method uses prior knowledge to extract a small amount of feature information of the ideal reference image, compares it with the feature information of the distorted image, and completes the quality assessment of the distorted image. The non-reference image quality assessment method NR-IQA evaluates the quality of a distorted image without any reference image, which is more suitable for practical application scenarios.

[0007] Traditional NR-IQA methods use the perceptual characteristics of the human visual system (HVS) to obtain useful features from the image to be evaluated for modeling, and map the degree of image distortion to a quality score; for example, Liu et al. proposed the SNP-NIQE algorithm, which extracts the natural statistical characteristics of distorted images from three aspects: structure, naturalness, and perception, and combines unsupervised learning for image quality assessment; Li et al. proposed using histograms to represent brightness statistical features and structural statistical features to perceive image quality changes, called NRSL, and used rotation-invariant local binary patterns to extract structural features.

[0008] In summary, the existing technologies for water meter image quality assessment have the following technical problems: (1) Subjective image quality assessment methods rely on human subjective feelings to judge the quality of images, which makes it difficult to achieve accurate and real-time quality assessment and is not suitable for large-scale water meter image data processing and automation systems; (2) Objective image quality assessment methods such as FR-IQA and RR-IQA mainly target specific distortion types, such as compression distortion, block effect distortion, and image blur, etc., and cannot adapt to the diversity of image distortion in actual water meter application scenarios; (3) The traditional no-reference image quality assessment NR-IQA method relies on manually designed features for training. This method has the limitations of low model accuracy and weak generalization performance, because the manually designed features may not be able to fully capture the complex features of the image, resulting in inaccurate evaluation results. Summary of the invention

[0009] 1. Technical issues to be solved

[0010] In view of the shortcomings of the prior art, the present invention provides an automatic evaluation method for smart water meter image quality based on deep learning, which can solve the above technical problems.

[0011] (II) Technical solution

[0012] In order to solve the above technical problems, the present invention provides the following technical solution: a method for automatically evaluating the quality of smart water meter images based on deep learning, comprising the following steps:

[0013] S1: Collect water meter images to make a dataset;

[0014] S2: extracting features from the water meter image using a first feature extraction module to obtain a first feature map;

[0015] S3: Inputting the first feature map into a second feature extraction module for feature extraction to obtain a second feature map;

[0016] S4: Map the first feature map and the second feature map to predict a quality evaluation score of the water meter image.

[0017] Preferably, the data set includes a training set, a validation set and a test set.

[0018] Preferably, step S1 further includes: performing data preprocessing on the water meter images; and performing data labeling on the water meter images in the training set and the validation set.

[0019] Preferably, the first feature extraction module is a convolutional neural network.

[0020] Preferably, the convolutional neural network includes four convolutional blocks.

[0021] Preferably, the second feature extraction module is a Transformer-based encoder.

[0022] Preferably, the Transformer-based encoder includes a multi-head self-attention layer and a feed-forward neural network.

[0023] Preferably, step S4 specifically includes: using a fully connected layer to fuse the first feature map and the second feature map and map them to a quality evaluation score.

[0024] Preferably, in step S1, the collected water meter images include water meter images of different water meter models, different water meter brands, and different water meter installation environments.

[0025] Preferably, in step S1, the collected water meter images include water meter images with different distortion conditions.

[0026] (III) Beneficial effects

[0027] Compared with the prior art, the present invention provides an automatic evaluation method for smart water meter image quality based on deep learning, which has the following beneficial effects: (1) The present invention is a reference-free image quality evaluation method based on deep learning, which can achieve accurate and real-time quality evaluation and is suitable for large-scale water meter image data processing and automation systems; (2) The present invention uses a reference-free image quality evaluation model, which does not require a fixed reference image and is more suitable for actual application scenarios. This is different from the traditional reference image quality evaluation method, which requires the use of a corresponding reference image for comparison and evaluation of each type of water meter image, while the reference-free evaluation method of the present invention is more flexible and suitable for the diversity of image distortion situations in actual scenarios. It can better adapt to the image quality assessment needs under different environments and conditions; (3) The present invention can more accurately extract image information features through the first feature extraction module and the second feature extraction module, thereby improving the accuracy and practicality of image quality assessment prediction. Compared with the traditional NR-IQA image quality evaluation method that relies on manually designed features, the NR-IQA model based on deep learning of the present invention does not require manually designed features and can better capture the local and global complex features of the image, thereby improving the accuracy and robustness of image quality prediction, that is, improving the accuracy and generalization ability of the evaluation model, realizing the automated evaluation of the image quality of smart water meters, and detecting the image acquisition quality of smart water meter equipment, so as to further improve the accuracy of water meter image data recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a flowchart of the steps of the method for automatic assessment of smart water meter image quality based on deep learning of the present invention;

[0029] Figure 2 Some of the water meter image samples collected;

[0030] Figure 3 This is the structural diagram of the convolutional neural network;

[0031] Figure 4 This is the structural diagram of the Transformer-based encoder;

[0032] Figure 5 Comparison chart of quality assessment prediction results of some water meter images. DETAILED DESCRIPTION

[0033] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0034] The present invention provides an automatic evaluation method for smart water meter image quality based on deep learning. Specifically, the present invention is a no-reference image quality assessment NR-IQA method based on deep learning. The present invention comprises the following steps:

[0035] S1: Collect water meter images to create a dataset.

[0036] The water meter image can be specifically captured and collected by a micro camera device installed at the end of the water meter and uploaded to the server. The water meter image is allowed to contain some background information except the reading area; preferably, the collected water meter images include water meter images of different water meter models, different water meter brands, and different water meter installation environments; the collected water meter images also include water meter images of different distortion conditions to enrich the data diversity of the data set, so that the present invention can be better applied to the actual application scenarios of different water meter image quality assessment. Some of the collected water meter image samples are as follows: Figure 2 shown.

[0037] Step S1 also includes data preprocessing of the collected water meter images: selecting representative water meter images, covering water meter images of different water meter models, different water meter brands, and different water meter installation environments; and ensuring that the image data covers various typical distortions that affect the clarity and recognizability of the reading image, including blur, noise, occlusion, uneven lighting, scratches, stains, corrosion, and other different water meter images to construct a data set.

[0038] The data set includes a training set, a validation set and a test set. Specifically, the multiple water meter images collected can be divided into a training set, a validation set and a test set in a ratio of 8:1:1. It can be understood that the training set and the validation set are applied to the training stage of the reference-free image quality assessment model of the present invention: the training set is used for model training, the validation set is used for model validation to update model parameters; and the test set is used for model testing.

[0039] In addition, step S1 also includes data annotation of the water meter images in the training set and the validation set, that is, annotating the corresponding quality evaluation score on the water meter image: the data annotation can specifically adopt a subjective evaluation method to obtain a subjective quality evaluation score, and the observer observes different evaluation indicators of the quality attributes of the water meter images in the training set and the validation set, such as the clarity, contrast, occlusion blur degree and other evaluation indicators of the reading area to give a subjective score of the water meter image quality, and then calculates the mean subjective score (MOS, Mean Opinion Score) to obtain a comprehensive evaluation of the image quality (i.e., the above-mentioned subjective quality evaluation score), which provides a standard for the training and verification of the reference-free image quality assessment model of the present invention. The calculation formula of MOS is as follows:

[0040]

[0041] In the above formula, C is the number of observers, S xc is the score that the cth subject evaluates on the xth image under different indicators.

[0042] Next, the no-reference image quality assessment model of the present invention is trained using a training set and a validation set. The no-reference image quality assessment model includes a first feature extraction module, a second feature extraction module, and an image quality prediction module. The training process is described in the following steps S2-S4.

[0043] S2: Using a first feature extraction module to extract features from the water meter image to obtain a first feature map.

[0044] Preferably, the first feature extraction module is a convolutional neural network (CNN), and the structure of the convolutional neural network is as follows: Figure 3 As shown, CNN extracts multi-scale features from the input image through the structure of four convolution blocks. These features capture the local and global information of the image at different levels. In addition, in order to process the features extracted from different layers, CNN introduces feature normalization, pooling layer and dropout processing after the convolution block; dropout is a typical processing method to increase the robustness of the neural network model. During the training process, some feature vectors are randomly discarded to increase the learning ability of the model, thereby improving the robustness of the model; finally, CNN splices the features after dropout processing through the Concat operation to form the final output feature representation of CNN. The multi-scale feature fusion method of CNN can comprehensively utilize the feature information of different levels to provide a richer and more comprehensive feature representation for the reference-free image quality assessment model of the present invention.

[0045] S3: Input the first feature map into a second feature extraction module for feature extraction to obtain a second feature map.

[0046] Preferably, the second feature extraction module is a Transformer-based encoder, which is one of the core components in the Transformer model. The Transformer network model is a deep learning architecture proposed by Vaswan i et al. Generally speaking, traditional sequential models (such as recurrent neural networks (RNNs)) have limitations in capturing long-range dependencies and implementing parallel computing. In order to solve these problems, the Transformer model introduces a self-attention mechanism. By widely using this mechanism, the Transformer model can weigh the importance of different positions in the input sequence when generating outputs. The Transformer model can obtain a global view of the image, and it has a good feature extraction and representation capability for the image, thereby improving the recognition accuracy of the image; the Transformer model can better handle long-distance dependencies through the advantages of self-attention mechanism and parallel computing, and improve the training and reasoning efficiency of the model.

[0047] like Figure 4 As shown in the figure, the Transformer-based encoder includes a multi-head self-attention layer and a feedforward neural network. The encoder is used to perform attention operations on multi-scale extracted features from different layers and model the dependencies between them. Compared with traditional CNN, the Transformer architecture can overcome the local limitations of CNN and effectively capture the complex dependencies between image features. The multi-head self-attention mechanism adopted by the multi-head self-attention layer performs attention calculations on different subspaces. The multi-head self-attention mechanism enables the model to pay attention to information at different levels and different aspects in the input sequence, thereby better performing feature extraction and representation learning. The feedforward neural network consists of two linear layers and an activation function. The feedforward neural network is used to perform nonlinear transformation and mapping on the features at each position, thereby further improving the expressiveness of the model.

[0048] In step S3, the multi-scale features of the first feature map are first converted into linear vectors, and the linear vectors are input into the Transformer-based encoder; the non-local features of the first feature map are calculated using the multi-head self-attention layer to capture the information of each extracted feature element relative to other feature elements. Then, residual connections and normalization operations are added between each layer of the encoder, and the output in the multi-head self-attention layer is further processed by a feedforward neural network to improve its feature representation, and finally the second feature map containing feature information such as global features is output by the Transformer-based encoder. The present invention can obtain a global view of the water meter image through a Transformer-based encoder, which has a good feature extraction and characterization capability for the water meter image, thereby effectively improving the quality assessment accuracy of the water meter image.

[0049] S4: Map the first feature map and the second feature map to predict a quality evaluation score of the water meter image.

[0050] Preferably, the image quality prediction module of the reference-free image quality assessment model of the present invention adopts a fully connected layer. The step S4 is specifically as follows: using the fully connected layer to fuse the first feature map and the second feature map and map them to a quality evaluation score. The fully connected layer can specifically fuse the feature vectors of the first feature map and the feature vectors of the second feature map in a series manner. Then, the fully connected layer is used as a regression network to complete the regression mapping of the fused features to the quality evaluation score, that is, outputting the quality evaluation score predicted by the water meter image. The higher the quality evaluation score, the clearer the corresponding water meter image is. For example, Figure 5 As shown, the prediction results of the reference-free image quality assessment model of the present invention can distinguish between clear and easy-to-read water meter images and blurred and distorted water meter images that are difficult to read; the fully connected layer of the present invention is a prior art, and its completion of the mapping of image features to image quality scores will not be described in detail here.

[0051] In addition, after completing the training of the no-reference image quality assessment model of the present invention in the above step S4, the water meter images in the above test set can be further input into the trained no-reference image quality assessment model to predict and output the quality evaluation scores corresponding to each water meter image to be quality assessed.

[0052] Compared with the prior art, the present invention provides an automatic evaluation method for image quality of smart water meters based on deep learning, which has the following beneficial effects: (1) The present invention is a reference-free image quality evaluation method based on deep learning, which can realize accurate and real-time quality evaluation and is suitable for large-scale water meter image data processing and automation systems; (2) The present invention designs a reference-free image quality evaluation model based on deep learning technology, which does not require a fixed reference image and is more suitable for actual application scenarios. This is different from the traditional reference image quality evaluation method. The traditional image quality evaluation method requires the use of a corresponding reference image for comparison and evaluation of each type of water meter image, while the reference-free evaluation method of the present invention is more flexible and applicable to the diversity of image distortion in actual scenarios, and can better adapt to the image quality evaluation requirements under different environments and conditions; (3) The present invention can more accurately extract image information features through the first feature extraction module and the second feature extraction module, thereby improving the accuracy and practicality of image quality evaluation prediction. Compared with the traditional NR-IQA image quality evaluation method that relies on manually designed features, the NR-IQA model based on deep learning of the present invention does not require manually designed features and can It can better capture the local and global complex features of the image, thereby improving the accuracy and robustness of image quality prediction; (4) The feature extraction of the reference-free image quality assessment model of the present invention combines a convolutional neural network and a Transformer-based encoder to extract local features (convolutional neural network) and non-local features (i.e., global features, realized by the self-attention of the Transformer-based encoder) of the water meter image, and fuses local features and non-local features, so that it can more accurately extract image information, better capture the local and global features of the image, and output digital evaluation measurement results that are highly consistent with subjective evaluation, thereby improving the accuracy and robustness of image quality assessment prediction; (5) Compared with the NR-IQA method using two-stage training in the prior art, the two-stage training requires the design and adjustment of multiple stages of training processes, which increases the complexity of the training process. The scalability of the model is limited by the architecture of the two-stage training, and it is difficult to flexibly respond to different types of water meter image evaluation requirements; the reference-free image quality assessment model of the present invention adopts an end-to-end training method, which is more flexible in training and has stronger scalability, and is suitable for water meter images of various specifications and styles.

[0053] It should be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0054] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. The automatic assessment method of smart water meter image quality based on deep learning is characterized by: The following steps are involved: S1: Collect water meter images to make a dataset; S2: extracting features from the water meter image using a first feature extraction module to obtain a first feature map; S3: Inputting the first feature map into a second feature extraction module for feature extraction to obtain a second feature map; S4: Mapping the first feature map and the second feature map to predict a quality evaluation score of the water meter image.

2. According to the deep learning-based automatic assessment method for smart water meter image quality according to claim 1, it is characterized by: The data set includes a training set, a validation set and a test set.

3. The method for automatic assessment of smart water meter image quality based on deep learning according to claim 2 is characterized in that: The step S1 also includes: performing data preprocessing on the water meter images; and performing data labeling on the water meter images in the training set and the validation set.

4. The method for automatic assessment of smart water meter image quality based on deep learning according to claim 1 is characterized in that: The first feature extraction module is a convolutional neural network.

5. The method for automatic assessment of smart water meter image quality based on deep learning according to claim 4 is characterized in that: The convolutional neural network includes four convolution blocks.

6. The method for automatic assessment of smart water meter image quality based on deep learning according to claim 1 is characterized in that: The second feature extraction module is a Transformer-based encoder.

7. The method for automatic assessment of smart water meter image quality based on deep learning according to claim 6 is characterized in that: The Transformer-based encoder includes a multi-head self-attention layer and a feed-forward neural network.

8. The method for automatic assessment of smart water meter image quality based on deep learning according to claim 1 is characterized in that: The step S4 specifically includes: using a fully connected layer to perform feature fusion on the first feature map and the second feature map and map them to the quality evaluation score.

9. The method for automatic assessment of smart water meter image quality based on deep learning according to claim 1 is characterized in that: In the step S1, the collected water meter images include water meter images of different water meter models, different water meter brands, and different water meter installation environments.

10. The method for automatic assessment of smart water meter image quality based on deep learning according to claim 9 is characterized in that: In the step S1, the collected water meter images include water meter images with different distortion conditions.

Citation Information

Patent Citations

  • Online traveling wave fault identification method based on novel Transform architecture

    CN117972516A

Cited By

  • Image quality evaluation method and system

    CN120125975A