Method for improving full-life-cycle management level of metering assets based on deep learning and bar code recognition technology

Abstract

The invention discloses a method for improving the full-life-cycle management level of metering assets based on deep learning and a bar code recognition technology, and relates to the field of full-life-cycle management of metering assets. The method comprises the steps that firstly, a lightweight neural network is used for extracting image features, an asset number area at any angle under any background is rapidly positioned, in the area, text character strings are recognized through a sequence classifier, the speed is high, and the accuracy is high; meanwhile, the identification of metering equipment asset codes is realized in combination with a bar code decoding technology, so that the identification accuracy is improved while the rapid identification of the asset numbers is ensured; and finally, compiling a python program to realize batch identification and renaming of the picture set. Therefore, the full-life-cycle management level of the metering assets is improved.

Description

Technical Field

[0001] The present invention relates to a method for improving the full life cycle management level of metrological assets based on deep learning and barcode recognition technology, and belongs to the field of full life cycle management of metrological assets. Background Art

[0002] The metering asset number is the unique identification code of the metering equipment. It is a set of numbers assigned to each metering asset to identify it. Since the metering asset code is unique and is not easily tampered with when etched on the metering equipment, it is of great significance when recycling and sorting metering assets.

[0003] With the State Grid Corporation's emphasis on the full life cycle management of metering equipment, in recent years, in addition to requiring full recycling of all types of dismantled metering equipment, it is also required to establish a digital archive for each device, take photos, and store the photos named with asset codes. Taking the sorting of electric energy meters as an example, currently the staff uses a high-speed scanner to pick up the electric energy meters one by one and take photos, and rename the photos through the barcode recognition software that comes with the high-speed scanner. Each step requires manual operation, and the efficiency is slow. At the same time, due to the influence of factors such as the recognition angle, light, and dirt on the surface of the electric energy meter of the high-speed scanner, the accuracy of electric energy meter asset recognition is also low, and often requires a lot of manual intervention, which is time-consuming and labor-intensive. Therefore, the present invention proposes a method for improving the level of full life cycle management of metering assets based on deep learning and barcode recognition technology to solve the above problems. Summary of the invention

[0004] According to the problem described in the background, the present invention proposes a method for image recognition of asset numbers of measuring equipment in complex scenarios based on neural network and barcode recognition technology. First, a lightweight neural network is used to extract image features to quickly locate the asset number area at any angle and in any background. In this area, a sequence classifier is used to recognize text strings, which is not only fast but also accurate. At the same time, the barcode decoding technology is combined to realize the recognition of the asset code of the measuring equipment, ensuring the rapid recognition of the asset number while improving the recognition accuracy. Finally, the accuracy of the recognition result is improved through the credibility label, and a python program is written to realize batch recognition and renaming of the picture set.

[0005] To achieve the above object, the present invention provides a method for improving the management level of the whole life cycle of metrological assets based on deep learning and barcode recognition technology, which is characterized by comprising the steps of:

[0006] S1. Extract asset codes using convolutional neural networks: Use lightweight neural networks to extract image features and quickly locate the asset number area at any angle and in any background. In this area, use sequence classifiers to recognize text strings.

[0007] S2. Extract asset code by barcode recognition: Combine barcode decoding technology to realize the recognition of asset code of metering equipment;

[0008] S3. Improve the accuracy of recognition results through credibility labels: Write a Python program to achieve batch recognition and renaming of image sets.

[0009] Among them, the convolutional neural network in step S1 extracts the asset code by using a lightweight neural network to extract image features, that is, the recognition of the asset number is regarded as a sequence-related classification, and other marginal characters that do not need to be recognized are directly ignored. The backbone network structure of the convolutional neural network in step S1 for extracting the asset code for measuring equipment asset number feature extraction is VGGNet, and a 3×3 convolution kernel is used to extract image features. The feature pooling layer in the convolutional neural network in step S1 for extracting the asset code is the Maxpool layer, so that the sampling result is sampled on a 2x image, and the network uses rounding down when downsampling. The loss function used to calculate the VIN recognition network in the convolutional neural network in step S1 for extracting the asset code is: and M is the total length of the asset number of the measuring equipment, which is set to 22; N is all the categories of characters, which is set to 33; for the kth character, Predicted value y k and the true label The cross entropy loss, L rec is the sum of the losses of all classifiers.

[0010] Furthermore, in the process of extracting asset codes by the convolutional neural network in step S2, when training the detection network, the training image is rotated in any direction to improve the effect of detecting multi-angle images, and the original image and the quadrilateral text box label are rotated using a python function. The software package imported in step S2 barcode recognition and asset code extraction uses NumPy for numerical calculations, argparse for parsing command line parameters, and OpenCv for image recognition. The step S2 of barcode recognition and asset code extraction requires loading an image from a disk and converting it into a grayscale image. After using the Scharr operation, the gradient amplitude representation of the grayscale image in the horizontal and vertical directions is constructed. After the Scharr operation, the y-gradient is subtracted from the x-gradient. Through this subtraction operation, an image region containing a high horizontal gradient and a low vertical gradient is finally obtained; the regional image is subjected to perspective transformation for tilt correction, and then the focus is focused on the barcode region through image denoising. First, a 9*9 kernel is used to average blur the gradient image, and then the blurred image is binarized. Any pixel less than or equal to 255 in the gradient image is set to 0 (black), and the rest are set to 255 (white); morphological operations are used to eliminate the gaps between the verticals in the barcode, so that the algorithm can more easily detect the "spotted" area in the barcode. Four corrosions and four dilations are used to eliminate small spots, and the barcode area can be detected and identified by the algorithm.

[0011] Furthermore, the step S3 improves the accuracy of the recognition result by using the credibility label to compare the asset code extracted by the convolutional neural network with the asset code extracted by the barcode recognition, and obtains the comparison result of the two extraction code methods through the three judgment items of string length, whether it contains text, and string prefix matching. In the step S3, the asset code extracted by the convolutional neural network and the asset code extracted by the barcode recognition are used to improve the accuracy of the recognition result by using the credibility label. When the three judgment items in the two results are correct, the algorithm will give priority to the text recognition result of the convolutional neural network as the output result; when the three judgment items in one of the two results are correct, the algorithm will select this result as the output result; when there are cross-errors in the three judgment items in the two results, the algorithm will select the correct judgment item to merge the results and output the results to the database that needs manual verification; when all the three judgment items in the two results are wrong, the algorithm will not output the result and hand it over to manual recognition.

[0012] The beneficial effects of the present invention are:

[0013] In the process of extracting asset codes by convolutional neural networks, lightweight neural networks extract image features. In this area, the sequence classifier is used to recognize text strings, which can improve the speed and accuracy of extracting asset codes. Using VGGNet trunk and head structure to extract features can ensure the robustness of image text recognition and avoid the problem of low recognition rate caused by shooting angles. Using barcodes to extract asset codes can quickly and effectively identify the asset codes of metering equipment in pictures. After that, the accuracy of recognition results is improved by using credibility labels. The results of the two extraction methods are analyzed and compared, which can quickly extract asset codes with higher accuracy. Compared with the current method, the time efficiency is improved by about 95%, and the recognition accuracy is improved by 5%. It can effectively solve the problem of time-consuming and labor-intensive manual operations, improve labor productivity, and improve the management level of the entire life cycle of metering assets. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 This is a flow chart of a method for improving the management level of the entire life cycle of metrological assets based on deep learning and barcode recognition technology involved in the present invention;

[0015] Figure 2 VGGNet is the backbone network structure used for asset number feature extraction of metering equipment. DETAILED DESCRIPTION

[0016] The embodiments of the present invention are further described below in conjunction with the accompanying drawings:

[0017] In order to verify the effectiveness of the results of the present invention, a dataset of 15,000 photos taken by area managers was manually collected - the disassembled electric energy meter dataset. Each electric energy meter was not wiped clean of dirt on the surface and was kept original to test the recognition accuracy and recognition speed of the method under normal circumstances.

[0018] Among them, the 15,000 photos of electricity meters include 3,000 clear photos taken at normal angles, 3,000 photos of electricity meters taken at abnormal angles but with clear images, 3,000 photos of electricity meters taken at normal angles but with damaged barcodes, 3,000 photos of electricity meters with damaged barcodes and asset codes, and 3,000 photos of electricity meters with blurred focus and shaking. Experiments were conducted on the above pictures to extract asset codes using convolutional neural networks, extract asset codes using barcode recognition, and improve the accuracy of recognition results through credibility labels.

[0019] Among them, for different photos, the first time the asset code is extracted using a convolutional neural network, the image is first identified to detect whether the shooting angle is normal. If it is normal, only cropping is performed. If it is not normal, it is rotated first and then cropped. After cropping, other edge characters that are not wanted to be recognized are ignored. Secondly, the backbone network structure VGGNet is used to extract image features, and the loss function of the VIN recognition network is calculated. The python function is used to rotate the original image and the quadrilateral text box label at the same time, and finally the first extraction result is obtained; the second time the asset code is extracted, the barcode recognition extracts the asset code, NumPy is used for numerical calculation, argparse is used to parse command line parameters, OpenCv is used for image recognition, the image is loaded from the disk and converted to a grayscale image, and the Scharr operation is used to construct the gradient amplitude representation of the grayscale image in the horizontal and vertical directions. After the Scharr operation, the y-gradient is subtracted from the x-gradient. Through this subtraction operation, the image area containing high horizontal gradient and low vertical gradient is finally obtained. The regional image is subjected to perspective transformation for tilt correction, and then the focus is focused on the barcode area through image denoising. First, the gradient image is averaged and blurred using a 9*9 kernel, and then the blurred image is binarized. Any pixel less than or equal to 255 in the gradient image is set to 0 (black), and the rest are set to 255 (white). Morphological operations are used to eliminate the gaps between the vertical lines in the barcode, making it easier for the algorithm to detect the "spotted" area in the barcode. Using 4 corrosions and 4 dilations to eliminate small spots, the algorithm can detect the barcode area and identify it.

[0020] Finally, the accuracy of the recognition results is improved through credibility labels: the asset code extracted by the convolutional neural network is compared with the asset code extracted by the barcode recognition, and the credibility labels are set for the results output by the two methods through the three judgment items of string length, whether it contains text, and string prefix matching. When the three judgment items of the two results are all correct, the algorithm will give priority to the text recognition result of the convolutional neural network as the output result; when one of the three judgment items of the two results is correct, the algorithm will select this result as the output result; when there are cross-errors in the three judgment items of the two results, the algorithm will select the correct judgment item to fuse the results and output the results to the database that needs manual verification. When all three judgment items of the two results are wrong, the algorithm will not output the result and hand it over to manual recognition.

[0021] After the above test verification, the following results were obtained:

[0022] 1. The recognition accuracy rate of photos with normal angles, no dirt on the surface, no shaking and overexposure is as high as 99.99%;

[0023] 2. Experimental tests on photos taken at tilted angles showed that because both algorithms corrected tilted photos before recognition, the recognition accuracy was not much different from that of normal photos, and the accuracy was higher;

[0024] 3. The experimental results for the case where the barcode is damaged are as follows: When the barcode is damaged, the accuracy of barcode recognition is seriously affected, while the convolutional neural network text recognition is not affected. By judging the weights of the two results, the algorithm selects the convolutional neural network recognition result with a higher accuracy.

[0025] 4. When both the barcode and the asset code are damaged, both methods will be seriously affected, but by judging the two results, the accuracy of image recognition can be improved;

[0026] 5. Photo blur and jitter have a greater impact on the barcode recognition accuracy, but a relatively small impact on the convolutional neural network recognition accuracy. After the algorithm integrates the results, a relatively ideal recognition accuracy can be guaranteed overall.

[0027] Finally, in terms of recognition efficiency, thanks to the lightweight convolutional neural network and optimized barcode recognition algorithm, the program takes about 1.25 hours to traverse 10,000 pictures and output the results. The time fluctuation is related to the picture quality, but the difference is very small and almost negligible. The average time to complete the recognition of each picture is about 0.5 seconds. The experimental results show that the method proposed in the present invention can quickly and effectively identify the asset code of the metering equipment in the picture, which is about 95% faster than the currently used method, and the recognition accuracy is improved by 5%. It can effectively solve the problem of time-consuming and labor-intensive manual operation, improve labor productivity, and improve the management level of the entire life cycle of metering assets.

Claims

1. A method to improve the management level of the whole life cycle of metrological assets based on deep learning and barcode recognition technology. It is characterized in that Includes steps: S1. Extract asset codes using convolutional neural networks: Use lightweight neural networks to extract image features and quickly locate the asset number area at any angle and in any background. In this area, use sequence classifiers to recognize text strings. S2. Extract asset code by barcode recognition: Combine barcode decoding technology to realize the recognition of asset code of metering equipment; S3. Improve the accuracy of recognition results through credibility labels: Write a Python program to achieve batch recognition and renaming of image sets.

2. According to claim 1, a method for improving the management level of the entire life cycle of measurement assets based on deep learning and barcode recognition technology, It is characterized in that In step S1, the convolutional neural network extracts the asset code by using a lightweight neural network to extract image features, that is, the recognition of the asset number is regarded as a sequence-related classification, and other marginal characters that do not need to be recognized are directly ignored.

3. According to claim 1, a method for improving the management level of the entire life cycle of measurement assets based on deep learning and barcode recognition technology, It is characterized in that The main network structure of the convolutional neural network in step S1 for extracting asset codes for measuring equipment asset number feature extraction is VGGNet, and a 3×3 convolution kernel is used to extract image features.

4. According to claim 1, a method for improving the management level of the entire life cycle of measurement assets based on deep learning and barcode recognition technology, It is characterized in that In the step S1, the feature pooling layer in the convolutional neural network for extracting the asset code is a Maxpool layer, so that the sampling result is sampled on a 2-fold image, and the network adopts rounding down when downsampling.

5. According to claim 1, a method for improving the management level of the entire life cycle of measurement assets based on deep learning and barcode recognition technology, It is characterized in that The loss function used to calculate the VIN recognition network in the step S1 convolutional neural network to extract the asset code is: and M is the total length of the asset number of the measuring equipment, which is set to 22; N is all the categories of characters, which is set to 33; for the kth character, Predicted value y k and the true label The cross entropy loss, L rec is the sum of the losses of all classifiers.

6. According to claim 1, a method for improving the management level of the entire life cycle of measurement assets based on deep learning and barcode recognition technology, It is characterized in that In the process of extracting asset codes by the convolutional neural network in step S2, when training the detection network, the training image is rotated in any direction to improve the effect of detecting multi-angle images, and the original image and the quadrilateral text box label are rotated using a python function.

7. According to claim 1, a method for improving the management level of the entire life cycle of measurement assets based on deep learning and barcode recognition technology, It is characterized in that The software package imported in step S2 for barcode recognition and asset code extraction uses NumPy for numerical calculations, argparse for parsing command line parameters, and OpenCv for image recognition.

8. According to claim 1, a method for improving the management level of the entire life cycle of metrological assets based on deep learning and barcode recognition technology, It is characterized in that The step S2 of barcode recognition and asset code extraction requires loading an image from a disk and converting it into a grayscale image. After the Scharr operation, the gradient amplitude representation of the grayscale image in the horizontal and vertical directions is constructed. After the Scharr operation, the y-gradient is subtracted from the x-gradient. Through this subtraction operation, an image region containing a high horizontal gradient and a low vertical gradient is finally obtained; the regional image is subjected to perspective transformation for tilt correction, and then the focus is focused on the barcode region through image denoising. First, the gradient image is averaged and blurred using a 9*9 kernel, and then the blurred image is binarized. Any pixel less than or equal to 255 in the gradient image is set to 0 (black), and the rest is set to 255 (white); morphological operations are used to eliminate the gaps between the verticals in the barcode, so that the algorithm can more easily detect the "spot"-like area in the barcode. Four corrosions and four dilations are used to eliminate small spots, and the barcode area can be detected and identified by the algorithm.

9. According to claim 1, a method for improving the management level of the entire life cycle of measurement assets based on deep learning and barcode recognition technology, It is characterized in that The step S3 improves the accuracy of the recognition result by using the credibility label to compare the asset code extracted by the convolutional neural network with the asset code extracted by the barcode recognition, and obtains the comparison result of the two extraction code methods through three judgment items: string length, whether it contains text, and string prefix matching.

10. According to claim 1, a method for improving the management level of the entire life cycle of metrological assets based on deep learning and barcode recognition technology, It is characterized in that In the step S3, the accuracy of the recognition result is improved by using the credibility label. When the three judgment items in the two results are correct, the algorithm will give priority to the text recognition result of the convolutional neural network as the output result; when the three judgment items in one of the two results are correct, the algorithm will select this result as the output result; when there are cross errors in the three judgment items in the two results, the algorithm will select the correct judgment item to merge the results and output the results to the database that needs manual verification; When all three judgment items in two results are wrong, the algorithm will not output the results and will be handed over to manual identification.

Images