A method and system for intelligent recognition of digital instrument readings

Abstract

This invention discloses an intelligent method and system for recognizing digital instrument readings. The system employs an instrument positioning and reading recognition approach, and is an improvement upon the YOLOv5 target detection model. First, it identifies the location area of ​​digital instruments such as gas meters or electricity meters in an image. Then, it identifies the readings inside the instruments. During model training, an innovative heuristic training enhancement framework is proposed, and the model structure and reading recognition method are improved, resulting in both increased accuracy and faster recognition speed. This method is applicable to the recognition of any digital instrument, requiring only an additional fixed or similarly mounted mobile camera for image capture, and is robust to angle and background variations.

Description

Technical Field

[0001] This invention relates to the field of instrument recognition technology, and specifically to a method and system for intelligent recognition of digital instrument readings. Background Technology

[0002] Existing methods for identifying digital reading instruments such as gas meters and electricity meters typically consist of two parts: instrument location and digital reading recognition. Instrument location employs machine learning methods with manual feature extraction or deep learning object detection frameworks such as YOLO, Faster R-CNN, and Mask R-CNN. Digital reading recognition, however, mostly relies on OCR or traditional object detection methods. Manual feature extraction methods have poor generalization capabilities and are prone to experience bias. OCR methods attempt to perform semantic recognition on the entire reading string, but numbers, unlike text, do not possess contextual meaning, thus requiring no whole-block recognition. Furthermore, in limited real-world datasets, the information complexity of the OCR model itself is often insufficient for training and recognition. Traditional object detection frameworks can only be applied to general, relatively ideal scenarios, such as large publicly available datasets, and cannot be customized and optimized for specific business scenarios. Summary of the Invention

[0003] To address this, the present invention provides a method and system for intelligent identification of digital meter readings, which solves the problems of poor generalization ability, susceptibility to experience bias, inability to be customized and optimized for specific business scenarios, and inaccurate identification results in existing methods for identifying digital meter readings such as gas meters and electricity meters.

[0004] To achieve the above objectives, the present invention provides the following technical solution:

[0005] According to a first aspect of the present invention, a method for intelligent recognition of digital instrument readings is provided, the method comprising:

[0006] The feature extraction network is used to extract features from the collected raw instrument images to obtain the feature vector of each image sample, and an anomaly detection algorithm is used to filter out abnormal samples.

[0007] The remaining filtered samples are clustered using the k-means clustering algorithm. The same number of candidate sample clusters are divided according to the number of samples to be selected. One sample is randomly selected from each cluster to form a sample set. The readings in the dashboard of the sample set are labeled with rectangular boxes to obtain the training set.

[0008] The improved target detection model is trained using the training set. The target detection model identifies readings containing numbers and decimal points on the dashboard. The clustering feature construction is optimized and the identified numbers and decimal points are clustered using the k-means clustering algorithm. The numbers and decimal points in the same cluster are sorted according to their coordinates to obtain the correct reading.

[0009] Furthermore, the readings within the dashboard of the sample set are labeled with rectangular boxes, specifically including:

[0010] When labeling sample images, the images are first input into a pre-trained object detection model to obtain the target prediction boxes and confidence scores. If the confidence score of a prediction box for a certain number category in an image is lower than a set threshold, the sample is marked as an image that needs to be manually relabeled, thus automatically filtering images that need to be manually labeled.

[0011] Furthermore, the method also includes:

[0012] Improved mosaic data augmentation is performed on the training samples. First, the proportion of each number in each image is calculated based on the labeled data. After each training round, the contribution proportion of each number category to the loss function is calculated. When the contribution proportion of a certain number category is lower than the preset threshold, when performing improved mosaic data augmentation on the samples before the next training round, images with a relatively high proportion of the corresponding category are selected for stitching.

[0013] Furthermore, feature extraction is performed on the collected raw instrument images using a feature extraction network, specifically including:

[0014] The feature extraction network adopts the SPP-Inception feature extraction network, which replaces the single pooling layer in the Inception-v3 network with a spatial pyramid pooling network, and converts the input image into a 2048-dimensional feature vector.

[0015] Furthermore, anomaly detection algorithms are used to filter out anomalous samples, specifically including:

[0016] The anomaly detection algorithm employs the LSO-iForest anomaly detection algorithm to remove global and local anomaly data, specifically including:

[0017] PCA is used to reduce the dimensionality of the global features, and the corresponding dimensions with smaller feature values ​​are removed. The dimensions after dimensionality reduction are automatically selected through a certain strategy. Finally, the binary tree iTree is constructed by selecting and partitioning the dimensionality-reduced features.

[0018] Calculate the average height of each sample relative to its nearest sample. When the difference between the average height of a sample and the average height of its k surrounding samples in the tree exceeds a preset threshold, the sample is identified as an outlier and removed.

[0019] Furthermore, PCA dimensionality reduction is performed on the global features, removing dimensions with smaller feature values. The resulting dimensions are automatically selected using a specific strategy:

[0020] Let the sample feature matrix be X mxn Where n corresponds to the 2048 dimensions of the feature extraction network output, the covariance matrix A corresponding to X is calculated. nxn Performing EVD decomposition on A, we get A = QΣQ T Q is an orthogonal matrix composed of eigenvectors of A, where the magnitude of the eigenvalues ​​in Σ corresponds to the importance of the feature dimension of X.

[0021] W is formed by taking the eigenvectors corresponding to the k largest eigenvalues. nxk Then the dimension-reduced matrix X p =X nxk =X mxn ·W nxk The idea behind choosing k is based on the dimensionality-reduced feature matrix X. p The ratio of the sum of the standard deviations of each dimension to the original matrix exceeds a certain threshold q:

[0022]

[0023] Find the smallest k value that satisfies the conditions.

[0024] Furthermore, the average height of each sample relative to its nearest sample is calculated. When the difference between the average height of a sample and the average height of its k surrounding samples in the tree exceeds a preset threshold, the sample is identified as an outlier and removed. Specifically:

[0025]

[0026] Where x is the input sample, n is the number of samples in each binary tree iTree, and E(h(x)) is the average depth of x in the iTree, [x1, x2, ... x k Let ] represent the k nearest sample points to x; c(n) represents the average path length when the search in the binary search tree fails, where n represents the number of external nodes in the expanded binary search tree; the calculation and derivation of c(n) are as follows:

[0027]

[0028] The proof is as follows:

[0029] Take a sequence x = [x1, x2, ... xn A binary search tree contains a random permutation of a set of elements with insertion order x, where x1 = ... <x2<...<x n Let S n Let represent the average number of comparisons required for a successful search in a binary search tree.

[0030]

[0031] Pr represents x j It is x i The probability of an ancestor node; consider if x j It is x i The ancestor, that is, in (x i ,...x j ) or (x j ,...x i ), x j It is inserted first; because if x exists... k ∈(x i ,...x j If x is inserted first, then x i and x j It will be assigned to one of the two subtrees of that node, therefore:

[0032]

[0033] Because S n = (I+n) / n, where I represents the length of the internal path of the extended binary tree, therefore:

[0034] I = nS n -n=2(n+1)H n -4n

[0035] Therefore, the length of the external path of the extended binary tree is E = I + 2n = 2(n + 1)H n -2n, where u is the average number of comparisons u when the binary search tree fails. n =E / (n+1)=2H n -2n / (n+1), where n is the number of internal nodes. In LSO-iForest, n is the number of external nodes, so the number of internal nodes is n-1. Therefore, C(n) = u n-1 =2H n-1 -2(n-1) / n;

[0036] After the algorithm calculates, each sample has an outlier S(x,n,k). The samples with the largest outlier score are deleted according to a certain proportion.

[0037] Furthermore, the method also includes:

[0038] The improved target detection model further improves computational performance by replacing the residual network module in the YOLOv5 network model, which uses a residual network module-based CSP structure, with a lightweight Shuffle network module and introducing depthwise separable convolution, where each convolution kernel is responsible for only one channel.

[0039] Furthermore, the clustering feature construction is optimized, and the k-means clustering algorithm is used to cluster the identified digits and decimal points. The digits and decimal points in the same cluster are sorted according to their coordinates to obtain the correct reading. Specifically, this includes:

[0040] The y-axis feature weights of the numbers in the readings are amplified to increase the distance between numbers in different rows. Combined with KMeans and silhouette coefficients, numbers in different rows are clustered into different clusters. The specific clustering feature construction method is as follows:

[0041] feature=[xcenter,(ycenter+height / 2)*5]

[0042] Where (xcenter, ycenter) represents the coordinates of the center point of each number or decimal point rectangle, and height represents the height of the number or decimal point rectangle;

[0043] After dividing the numbers into corresponding reading clusters, the numbers within the same cluster are sorted according to their x-axis horizontal coordinates to obtain the correct readings.

[0044] According to a second aspect of the present invention, a digital instrument reading intelligent recognition system is provided, the system comprising:

[0045] The training sample screening module is used to extract features from the collected raw instrument images using a feature extraction network to obtain the feature vector of each image sample, and to filter out abnormal samples using an anomaly detection algorithm.

[0046] The remaining filtered samples are clustered using the k-means clustering algorithm. The same number of candidate sample clusters are divided according to the number of samples to be selected. One sample is randomly selected from each cluster to form a sample set. The readings in the dashboard of the sample set are labeled with rectangular boxes to obtain the training set.

[0047] The model training and recognition module is used to train the improved target detection model using the training set, identify readings containing numbers and decimal points in the dashboard through the target detection model, optimize the clustering feature construction and use the k-means clustering algorithm to cluster the identified numbers and decimal points, and sort the numbers and decimal points in the same cluster according to their coordinates to obtain the correct readings.

[0048] The present invention has the following advantages:

[0049] This invention proposes an intelligent method and system for recognizing digital instrument readings. The overall approach combines instrument positioning with reading recognition, and it improves upon the YOLOv5 target detection model. First, it identifies the location area of ​​digital instruments such as gas meters or electricity meters in an image. Then, it identifies the readings inside the instrument. During model training, an innovative heuristic training enhancement framework is proposed, and the model structure and reading recognition method are improved, resulting in both increased accuracy and faster recognition speed. This method is applicable to the recognition of any digital instrument, requiring only an additional fixed or similarly mounted mobile camera for image capture, and it is robust to angle and background variations. Attached Figure Description

[0050] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely exemplary, and those skilled in the art can derive other embodiments based on the provided drawings without creative effort.

[0051] Figure 1 This is a flowchart illustrating a digital instrument reading intelligent recognition method provided in Embodiment 1 of the present invention;

[0052] Figure 2 This is a detailed implementation process diagram of a digital instrument reading intelligent recognition method provided in Embodiment 1 of the present invention;

[0053] Figure 3 This is a schematic diagram comparing the modified SPP-Inception network with the original network in the intelligent recognition method for digital instrument readings provided in Embodiment 1 of the present invention;

[0054] Figure 4 This is a comparison diagram of depthwise separable convolution and ordinary convolution in a digital instrument reading intelligent recognition method provided in Embodiment 1 of the present invention;

[0055] Figure 5 This is a schematic diagram illustrating the structure and usage of the Shuffle network module in a digital instrument reading intelligent recognition method provided in Embodiment 1 of the present invention;

[0056] Figure 6 This is a diagram showing the target detection model recognition result in a digital instrument reading intelligent recognition method provided in Embodiment 1 of the present invention. Detailed Implementation

[0057] The following specific embodiments illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0058] Example 1

[0059] like Figure 1 As shown in the figure, this embodiment proposes a method for intelligent recognition of digital instrument readings, the method comprising:

[0060] S100. Use a feature extraction network to extract features from the collected raw instrument images to obtain the feature vector of each image sample, and use an anomaly detection algorithm to filter out abnormal samples.

[0061] S200. Use the k-means clustering algorithm to cluster the remaining filtered samples. Divide the sample into the same number of candidate sample clusters according to the number of samples to be selected. Randomly select one sample from each cluster to form a sample set. Mark the readings in the dashboard of the sample set with rectangular boxes to obtain the training set.

[0062] S300. The improved target detection model is trained using the training set. The target detection model identifies readings containing numbers and decimal points in the dashboard. The clustering feature construction is optimized and the k-means clustering algorithm is used to cluster the identified numbers and decimal points. The numbers and decimal points in the same cluster are sorted according to their coordinates to obtain the correct readings.

[0063] The details are as follows:

[0064] Current technologies for digital meter recognition mostly consist of combinations of general algorithms, failing to consider various real-world business scenarios. This invention proposes a heuristic-trained, enhanced neural network-based intelligent recognition method for digital meter readings. The method employs a combination of meter location and reading recognition, and is an improvement upon the YOLOv5 target detection model. First, the location area of ​​digital meters such as gas meters or electricity meters in the image is identified, followed by the recognition of the readings within the meters. During model training, an innovative heuristic training enhancement framework is proposed, and the model structure and reading recognition method are improved, resulting in both increased accuracy and faster recognition speed.

[0065] The heuristic training enhancement framework proposed in this invention mainly addresses three problems:

[0066] (1) How to select “high-value” training samples: In the early stages of a project, there is often no training data for the recognition task in a specific business scenario. It is necessary to collect data by camera at regular intervals and label it manually. However, the images transmitted back at regular intervals are of varying quality due to differences in lighting and angle, and they also have temporal correlation: that is, the differences between the instrument images collected in adjacent time periods or fixed periodic periods are small, and the changes in readings and background lighting are relatively slow. The discrimination is low when constructing sample features. Highly correlated images do not contribute enough to model training and are not conducive to improving the generalization ability and robustness of the model.

[0067] (2) How to solve the problem of insufficient training: During model training, the distribution of different categories of numbers in the image is uneven, which makes the contribution of some categories to the training loss of the model low, making it difficult for the model to learn the features of the category well; Existing image enhancement methods, such as mosaic data enhancement, only randomly rotate, scale, and crop the sample image and then stitch it together. Although the training dataset is expanded, this randomness does not take into account the specific type. Even if the amount of data is expanded, it still does not increase the contribution of certain categories to the training loss.

[0068] (3) How to reduce the manual cost of sample annotation: After a model has been trained based on a batch of labeled data, as new samples continue to be added, manual annotation of new samples is required continuously. When there are many numbers in a single meter image, the time spent annotating samples will increase significantly. Taking gas meters as an example, in actual projects, annotating the rectangular boxes and number categories of gas meter numbers, plus inspection time, takes about 2-4 minutes per image, that is, it takes about 5 hours to annotate 100 images. Deep learning models require a large number of labeled images, which greatly increases the time required for model iteration.

[0069] Heuristic training enhancement offers solutions to these problems:

[0070] 1. Feature extraction is performed on all collected original images using an improved SPP-Inception network. Each image is mapped to a feature vector. After removing some dirty data using an anomaly detection algorithm, a clustering algorithm is used to obtain a specified number of clusters. The number of clusters can be set to the number of samples the user wants to select. Then, one image is randomly selected from each cluster as a training sample. This greatly increases the variability between samples, improving sample diversity and model generalization ability compared to global random selection.

[0071] 2. To address the issue of insufficient contribution to the loss function during training for certain categories, an improved mosaic-like data augmentation method is adopted. Instead of random cropping and splicing, the use of the spliced ​​image is guided by the loss function during training, thereby improving the model's learning ability for these categories.

[0072] 3. Once a trained model is available, for the continuously collected instrument images, an Active Learning strategy can be employed. Images that "necessarily" require manual annotation are selected; that is, images that the model cannot clearly identify are deemed to require manual review. This approach allows for the start of the next round of model training with only a small number of manually reviewed samples, significantly reducing the cost of manual annotation and increasing the model's iteration speed.

[0073] In addition to the heuristic training enhancement framework, this invention also improves the model structure and reading recognition methods. The innovations of this invention can be summarized as follows:

[0074] 1. The SPP-Inception feature extraction network is introduced, and the LSO-iForest anomaly detection algorithm is proposed to remove global and local abnormal data. Finally, the KMeans clustering algorithm is used to divide the data and select high-value differential samples.

[0075] 2. Improve traditional mosaic data enhancement by guiding the use of mosaic images through the loss percentage of different categories;

[0076] 3. Introduce an Active Learning strategy based on object detection to automatically filter samples that require manual annotation based on confidence scores;

[0077] 4. Improve the network model structure by replacing the Residual Unit with a lightweight Shuffle Unit structure in the traditional YOLOv5 network model, further improving the model's computation speed and feature fusion capability;

[0078] 5. Optimize the construction of clustering features, and use KMeans and coordinate sorting algorithms to sort and classify the numbers and decimal points detected by the object detection model to obtain complete readings.

[0079] The overall flowchart of the model proposed in this embodiment of the invention is as follows: Figure 2 As shown. Specific details are as follows:

[0080] 1. The SPP-Inception feature extraction network is introduced, and the LSO-iForest anomaly detection algorithm is proposed to remove global and local anomalies. Finally, the KMeans clustering algorithm is used to segment the data and select high-value differential samples.

[0081] Faced with a massive amount of collected images, manual annotation is time-consuming and time-constrained, limiting the number of samples that can be annotated and used for training. The key challenge is selecting high-value, highly differentiated samples from a large pool of images for training and ensemble training, thus improving the model's generalization ability. The first step is to define a good set of "features" to distinguish images. Traditional image processing methods for extracting texture features can only express shallow localization features. However, with the development of deep learning, it has been proven that deeper networks can express more discriminative semantic features. Therefore, the first step is to design a robust feature extraction network to map images into a discriminative, high-dimensional feature space.

[0082] The traditional Inception-v3 network improves the model's expressive power by increasing network width. This is achieved by performing different degrees of convolution and pooling on the input data and concatenating the results. However, the pooling layers involved have limited expressive power, restricting the model's ability to abstract features from each layer. To address this issue, this invention introduces an improved SPP module, replacing the original single pooling layer in the Inception component. This more effectively increases the scope of backbone features and significantly separates the most important contextual features, as different contextual features are extracted into different channels. The modified SPP-Inception module is compared to the original... Figure 3 As shown.

[0083] By removing the fully connected layers used for classification, the SPP-Inception network can convert input images into 2048-dimensional feature vectors.

[0084] After obtaining the feature vectors of all captured images, a small amount of outlier data (images exhibiting halos or varying degrees of blurring due to shot noise, dark noise, and readout noise during camera capture) needs to be removed using an anomaly detection algorithm. A superior anomaly detection algorithm is Isolation Forest, which employs a decision tree-like approach, randomly selecting n samples as the root node of a tree each time, splitting the data at random dimensions and a certain threshold to construct T iTrees, and calculating the height of each sample within the T trees to characterize the degree of anomaly. This algorithm has linear time complexity and high execution efficiency, but iForest is not suitable for particularly high-dimensional data. Because the splitting is performed at random feature dimensions each time, a large number of dimensions may remain unutilized, and high-dimensional features themselves may contain some high-noise or low-discrepancy dimensions. Furthermore, iForest is only sensitive to global outliers and is not good at handling local relative outliers. This invention addresses these issues of iForest by proposing LSO-iForest. LSO stands for Local Space Outlier, which removes both global and local outliers.

[0085] Compared to the regular iForest, this method makes the following two improvements:

[0086] 1) Instead of randomly selecting and partitioning global features, the partitioning is performed using feature dimensions with "high differences". First, PCA dimensionality reduction is applied to the global features, removing dimensions with lower feature values. The resulting dimension k is automatically selected using a specific strategy. The calculation method is as follows: Let the sample feature matrix be X. mxn (Where n corresponds to the 2048 dimensions of the previous SPP-Inception output), calculate the covariance matrix A corresponding to X. nxn By performing EVD decomposition on A, we can obtain A = QΣQ T Let Q be an orthogonal matrix composed of eigenvectors of A. In this case, the magnitude of the eigenvalues ​​in Σ corresponds to the importance of the feature dimension of X. We take the eigenvectors corresponding to the k largest eigenvalues ​​to form W. nxk Then the dimension-reduced matrix X p =X nxk =X mxn ·W nxk The idea behind choosing k is based on the dimensionality-reduced feature matrix X. p The ratio of the sum of the standard deviations of each dimension to the original matrix exceeds a certain threshold:

[0087]

[0088] Find the smallest k value that satisfies the conditions. In this business scenario, because the feature dimensionality is high and it is only used as the basis for partitioning, q of 0.95 is sufficient. Finally, select the partitions from the dimensionality-reduced features to construct the iTree, which can effectively improve the model's utilization of features in high-dimensional cases and improve the effectiveness of feature partitioning.

[0089] 2) Improved calculation method for sample outlier scores. The standard iForest only considers the global average height of each sample, resulting in the removal of only global outliers. This invention improves upon this by calculating the average height of each sample relative to its nearest neighbor. When a sample's average height differs significantly from the average height of its k surrounding samples in the tree, that point is identified as an outlier. The specific calculation method is as follows:

[0090]

[0091] Where x is the input sample, n is the number of samples in each iTree, and E(h(x)) is the average depth of x in the iTree, [x1, x2, ... x k Let ] represent the k nearest sample points to x, and c(n) represent the average path length when the search in the binary search tree (BST) fails (where n represents the number of external nodes in the expanded BST). The calculation and derivation of c(n) are as follows:

[0092]

[0093] The proof is as follows: Take a sequence x = [x1, x2, ... x n ], a random permutation of a BST with insertion order x, and x1 <x2<...<x n Let S n Let represent the average number of comparisons required for a successful search in the BST.

[0094]

[0095] Pr represents probability. Consider if x j It is x i The ancestor, that is, in (x i ,...x j ) or (x j ,...x i ), x j It is inserted first; because if x exists... k ∈(x i ,...x j If x is inserted first, then x i and x j It will be assigned to one of the two subtrees of that node, therefore:

[0096]

[0097] Because S n = (I+n) / n, where I represents the length of the internal path of the extended binary tree, therefore:

[0098] I = nS n -n=2(n+1)H n -4n

[0099] Therefore, the length of the external path of the extended binary tree is E = I + 2n = 2(n + 1)H n -2n, where u is the average number of comparisons when the BST search fails. n =E / (n+1)=2H n -2n / (n+1). In this formula, n is the number of internal nodes, while in LSO-iForest, n is the number of external nodes, so the number of internal nodes is n-1. Therefore, C(n) = u n-1 =2H n-1 -2(n-1) / n. After calculation by the algorithm, each sample has an outlier S(x,n,k). The samples with the largest outlier score are deleted according to a certain proportion.

[0100] After anomaly filtering, KMeans clustering is performed on the feature vectors of the remaining samples. Compared to other clustering algorithms, KMeans is computationally faster and more suitable for scenarios with a large number of samples. Here, the value of k is the number of samples to be selected. For example, if 500 images need to be selected from 10,000 images, then only 500 clusters need to be formed, and one sample needs to be selected from each cluster. Compared to randomly selecting from all images, this method yields more diverse samples, broadens the model's "vision," and improves the model's generalization ability.

[0101] 2. Improve traditional mosaic data enhancement by guiding the use of mosaic patterns through the loss percentage of different categories.

[0102] Traditional mosaic data augmentation uses four images, randomly scaled, cropped, and arranged, to stitch them together to address the issues of limited data volume and uneven distribution of small, medium, and large targets. However, in instrument recognition scenarios, this random image selection and stitching method does not consider the uneven distribution of different digit categories in the readings. To address this issue, this embodiment of the invention considers selecting a stitching method beneficial to training based on a certain indicator. The most direct manifestation of uneven distribution of different categories is the proportion of the loss component for each digit category during each training round. When the contribution of certain classes to the loss is relatively small, image components containing these digit categories can be added to the stitched image.

[0103] The specific implementation involves first calculating the proportion of each number in each image based on the labeled data. After each training round, the contribution percentage of each category to the loss function is calculated. When the contribution percentage of a certain category falls below a certain threshold, during the mosaic data augmentation process before the next training round, images containing a higher proportion of those categories are selected for stitching. This heuristic "guided" stitching improves the model's training robustness and learning ability.

[0104] 3. Introduce an Active Learning strategy based on object detection to automatically filter samples that require manual annotation based on confidence scores.

[0105] This method aims to address the challenge of rapidly iterating model training. To reduce the time spent on manual annotation, methods are needed to help humans filter samples that the model can already accurately identify. This way, each time the model needs to be iterated, only a small number of samples need to be manually annotated for the next round of training—this is the general idea behind Active Learning. Specifically, in the application of instrument recognition, the model is first "pre-annotated," meaning it directly predicts using the model. For each digit and decimal point, the predicted bounding box outputs a confidence score, which is used to determine the accuracy of the prediction. When the confidence score of a predicted bounding box for a certain digit category in an image falls below a given threshold, that sample is marked as requiring manual re-annotation.

[0106] 4. Improve the network model structure by replacing the Residual Unit with a lightweight Shuffle Unit structure in the traditional YOLOv5 network model, further enhancing the model's computational speed and feature fusion capabilities.

[0107] In the object detection framework YOLOv5, the Backbone component employs a Residual Unit-based CSP structure. This component effectively extracts and fuses features to construct complex semantic features. However, a drawback is its relatively slow training speed. To improve training and inference speed, this invention proposes replacing the Residual Unit with a lightweight Shuffle Unit structure and introducing depthwise separable convolutions to further enhance computational performance.

[0108] Comparison between depthwise separable convolution and ordinary convolution Figure 4 As shown in the figure, each convolutional kernel in depthwise separable convolution is responsible for only one channel. Compared to the ordinary fully connected convolution, depthwise separable convolution further improves the computational speed of the model. Furthermore, the combination of depthwise convolution and pointwise convolution can improve computational speed with minimal loss of accuracy. The structure and usage of the Shuffle Unit are as follows... Figure 5As shown.

[0109] 5. Optimize the clustering feature construction, and use KMeans and coordinate sorting algorithms to sort and classify the digits and decimal points detected by the object detection model to obtain complete readings.

[0110] After the object detection model identifies the bounding boxes containing digits and decimal points, the recognition result is as follows: Figure 6 As shown.

[0111] Since the identified digit bounding boxes and category information are only individual digits or decimal points, it is necessary to group the digits and concatenate them into readings with different meanings. While clustering algorithms can be easily considered for this task, feature construction requires some skill. In practice, directly using the center coordinates (x, y) of each digit does not yield the desired clustering results. As can be seen in the figure, some digits have relatively close y-axis coordinates, causing adjacent digits to be clustered together, making it impossible to obtain meaningful readings.

[0112] To ensure that numbers are well distinguishable on both the x and y axes, observations of the actual data distribution reveal that the feature weight for the y-axis should be greater than that for the x-axis. This is because even slight differences along the y-axis indicate that numbers do not belong to the same cluster (readings are ordered by row). To improve model differentiation, the y-axis features of numbers not belonging to the same cluster are amplified, thereby increasing the distance between numbers in different rows. The specific feature construction method proposed in this embodiment is as follows:

[0113] feature=[xcenter,(ycenter+height / 2)*5]

[0114] Where (xcenter, ycenter) represents the coordinates of the center point of each digit or decimal point rectangle, and height represents the height of the digit or decimal point rectangle. Using this feature construction method, combined with KMeans and profile coefficients, it is relatively easy to classify digits into appropriate reading clusters.

[0115] After dividing the numbers into corresponding reading clusters, the numbers and decimal points within the same cluster need to be sorted by coordinates to obtain the correct reading. Since humans currently read number strings from left to right, line by line, the numbers within the same cluster can be sorted according to their x-axis horizontal coordinates.

[0116] This invention proposes an intelligent recognition method for digital meter readings using a heuristic-trained enhanced neural network. Utilizing deep learning object detection and other related improved technologies, it identifies digital meter readings such as gas meters and electricity meters, extracts their reading information, and automatically inputs it into an information system, reducing the cost of manual inspection and data collection. The technologies involved include: deep neural networks, heuristic training enhancement algorithms, Active Learning, object detection technology, clustering algorithms, and anomaly detection algorithms. This technology can be applied to the following aspects:

[0117] (1) In the intelligent inspection robot, the robot takes pictures of digital instruments such as gas meters or electricity meters, transmits the pictures back to the system, uses the method provided in this embodiment to identify them, and returns the identification results in real time, which are displayed on the web or stored in the database.

[0118] (2) For some older communities or companies with electricity meters that have not been upgraded to smart meters, electricity consumption can only be collected through manual inspection because replacing them all with smart meters is too costly. In this case, a miniature camera can be installed on each meter (the cost of the camera is only 1 / 10 of that of a smart meter) to take pictures at regular intervals and transmit them back to the identification system, which can then identify and record the data using the method proposed in this embodiment.

[0119] The method described in this invention is applicable to the identification of any digital instrument, requiring only an additional fixed or similarly mounted mobile camera on a robot for image capture, and is robust to angle and background variations. This embodiment optimizes the model by improving the neural network model structure and incorporating heuristic training enhancements, enabling the model to more accurately identify digital instrument readings.

[0120] Example 2

[0121] Corresponding to Embodiment 1 above, this embodiment proposes a digital instrument reading intelligent recognition system, the system comprising:

[0122] The training sample screening module is used to extract features from the collected raw instrument images using a feature extraction network to obtain the feature vector of each image sample, and to filter out abnormal samples using an anomaly detection algorithm.

[0123] The remaining filtered samples are clustered using the k-means clustering algorithm. The same number of candidate sample clusters are divided according to the number of samples to be selected. One sample is randomly selected from each cluster to form a sample set. The readings in the dashboard of the sample set are labeled with rectangular boxes to obtain the training set.

[0124] The model training and recognition module is used to train the improved target detection model using the training set, identify readings containing numbers and decimal points in the dashboard through the target detection model, optimize the clustering feature construction and use the k-means clustering algorithm to cluster the identified numbers and decimal points, and sort the numbers and decimal points in the same cluster according to their coordinates to obtain the correct readings.

[0125] The functions performed by each component in the intelligent identification system for digital instrument readings provided in this embodiment of the invention have been described in detail in the above embodiment 1, so they will not be repeated here.

[0126] Although the present invention has been described in detail above with general descriptions and specific embodiments, modifications or improvements can be made to it, which will be obvious to those skilled in the art. Therefore, all such modifications or improvements made without departing from the spirit of the present invention fall within the scope of protection claimed by the present invention.

Claims

1. A method for intelligent recognition of digital instrument readings, characterized in that, The method includes: Feature extraction is performed on the collected raw instrument images using a feature extraction network to obtain the feature vector for each image sample. Anomaly detection algorithms are then used to filter out abnormal samples. The anomaly detection algorithm for filtering abnormal samples specifically includes: The anomaly detection algorithm employs the LSO-iForest anomaly detection algorithm to remove global and local anomaly data, specifically including: PCA is used to reduce the dimensionality of the global features, and the corresponding dimensions with smaller feature values ​​are removed. The dimensions after dimensionality reduction are automatically selected through a certain strategy. Finally, the binary tree iTree is constructed by selecting and partitioning the dimensionality-reduced features. Calculate the average height of each sample relative to its nearest sample. When the difference between the average height of a sample and the average height of its k surrounding samples in the tree exceeds a preset threshold, the sample is identified as an outlier and removed. The remaining filtered samples are clustered using the k-means clustering algorithm. The same number of candidate sample clusters are divided according to the number of samples to be selected. One sample is randomly selected from each cluster to form a sample set. The readings in the dashboard of the sample set are labeled with rectangular boxes to obtain the training set. The improved target detection model is trained using the training set. The target detection model identifies readings containing numbers and decimal points in the dashboard. The clustering feature construction is optimized and the k-means clustering algorithm is used to cluster the identified numbers and decimal points. The numbers and decimal points in the same cluster are sorted according to their coordinates to obtain the correct readings. The clustering feature construction is optimized, and the k-means clustering algorithm is used to cluster the identified digits and decimal points. The digits and decimal points in the same cluster are sorted according to their coordinates to obtain the correct reading. Specifically, this includes: The y-axis feature weights of the numbers in the readings are amplified to increase the distance between numbers in different rows. Combined with KMeans and silhouette coefficients, numbers in different rows are clustered into different clusters. The specific clustering feature construction method is as follows: ， in This indicates the coordinates of the center point of the rectangle containing each number or decimal point, and height indicates the height of the rectangle containing the number or decimal point. After dividing the numbers into corresponding reading clusters, the numbers within the same cluster are sorted according to their x-axis horizontal coordinates to obtain the correct readings.

2. The intelligent recognition method for digital instrument readings according to claim 1, characterized in that, The readings within the dashboard of the sample set are labeled with rectangular boxes, specifically including: When labeling sample images, the images are first input into a pre-trained object detection model to obtain the target prediction boxes and confidence scores. If the confidence score of a prediction box for a certain number category in an image is lower than a set threshold, the sample is marked as an image that needs to be manually relabeled, thus automatically filtering images that need to be manually labeled.

3. The intelligent recognition method for digital instrument readings according to claim 1, characterized in that, The method further includes: Improved mosaic data augmentation is applied to the training samples: First, the proportion of each number in each image is calculated based on the labeled data. After each round of training, the contribution of each number category to the loss function is calculated. When the contribution of a certain number category is lower than the preset threshold, the images with a relatively high proportion of the corresponding category are selected for stitching when the improved mosaic data augmentation is applied to the samples before the next round of training.

4. The intelligent recognition method for digital instrument readings according to claim 1, characterized in that, Feature extraction is performed on the collected raw instrument images using a feature extraction network, specifically including: The feature extraction network adopts the SPP-Inception feature extraction network, which replaces the single pooling layer in the Inception-v3 network with a spatial pyramid pooling network, and converts the input image into a 2048-dimensional feature vector.

5. The intelligent recognition method for digital instrument readings according to claim 1, characterized in that, PCA is used to reduce the dimensionality of the global features, removing dimensions with lower feature values. The resulting dimensions are then automatically selected using a specific strategy. Let the sample feature matrix be Where n corresponds to the 2048 dimensions of the feature extraction network output, and the covariance matrix corresponding to X is calculated. Perform EVD decomposition on A to obtain Q is an orthogonal matrix composed of the eigenvectors of A. The magnitude of the eigenvalue corresponds to the importance of the feature dimension of X; Take the eigenvectors corresponding to the first k largest eigenvalues ​​to form the eigenvalues. Then the dimension-reduced matrix The idea behind choosing k is based on the dimensionality-reduced feature matrix. The ratio of the sum of the standard deviations of each dimension to the original matrix exceeds a certain threshold. : ， Find the smallest k value that satisfies the condition.

6. The intelligent recognition method for digital instrument readings according to claim 1, characterized in that, Calculate the average height of each sample relative to its nearest neighbor. When the difference between the average height of a sample and the average height of its k neighboring samples in the tree exceeds a preset threshold, the sample is identified as an outlier and removed. Specifically: ， Where x is the input sample, and n is the number of samples in each binary tree iTree. Let x be the average depth in iTree. This represents the k closest sample points to x; Let c(n) represent the average path length when a search in the binary search tree fails, where n represents the number of external nodes in the expanded binary search tree. The calculation and derivation of c(n) are as follows: ， The proof is as follows: Take a sequence Given a binary search tree, a random permutation of a set of elements with insertion order x, and... ,set up Let represent the average number of comparisons required for a successful search in a binary search tree. ， Pr indicates The probability; consider if yes The ancestors, that is, in or In the subsequence of It was inserted first; because if it already existed... If it is inserted first, then and It will be assigned to one of the two subtrees of that node, therefore: ， because ,in This indicates the length of the internal path of the extended binary tree, therefore: ， Therefore, the external path length of the binary tree is extended. The average number of comparisons when a binary search tree fails. In the formula, n is the number of internal nodes, while in LSO-iForest, n is the number of external nodes. Therefore, the number of internal nodes is n-1. In summary, ; After algorithmic calculation, each sample has one outlier. The sample with the highest outlier score is deleted according to a certain ratio.

7. The intelligent recognition method for digital instrument readings according to claim 1, characterized in that, The method further includes: The improved target detection model further improves computational performance by replacing the residual network module in the YOLOv5 network model, which uses a residual network module-based CSP structure, with a lightweight Shuffle network module and introducing depthwise separable convolution, where each convolution kernel is responsible for only one channel.

8. A digital instrument reading intelligent recognition system, characterized in that, The system includes: The training sample filtering module is used to extract features from the collected raw instrument images using a feature extraction network to obtain the feature vector of each image sample, and to filter out abnormal samples using an anomaly detection algorithm. The anomaly detection algorithm for filtering out abnormal samples specifically includes: using the LSO-iForest anomaly detection algorithm to remove global and local abnormal data, specifically including: performing PCA dimensionality reduction on the global features, deleting dimensions with smaller corresponding feature values, automatically selecting the dimensions after dimensionality reduction using a certain strategy, and finally selecting and partitioning from the dimensionality-reduced features to construct a binary tree iTree; calculating the average height of each sample relative to its nearest sample, and when the difference between the average height of a sample and its k surrounding samples in the tree exceeds a preset threshold, the sample is judged as an anomaly and removed. The remaining filtered samples are clustered using the k-means clustering algorithm. The same number of candidate sample clusters are divided according to the number of samples to be selected. One sample is randomly selected from each cluster to form a sample set. The readings in the dashboard of the sample set are labeled with rectangular boxes to obtain the training set. The model training and recognition module is used to train the improved target detection model using the training set. The target detection model identifies readings containing numbers and decimal points on the dashboard. It optimizes clustering feature construction and uses the k-means clustering algorithm to cluster the identified numbers and decimal points. Numbers and decimal points in the same cluster are sorted according to their coordinates to obtain the correct reading. Specifically, this includes: The y-axis feature weights of the numbers in the readings are amplified to increase the distance between numbers in different rows. Combined with KMeans and silhouette coefficients, numbers in different rows are clustered into different clusters. The specific clustering feature construction method is as follows: ， in This indicates the coordinates of the center point of the rectangle containing each number or decimal point, and height indicates the height of the rectangle containing the number or decimal point. After dividing the numbers into corresponding reading clusters, the numbers within the same cluster are sorted according to their x-axis horizontal coordinates to obtain the correct readings.

Images