Automatic meter reading method of electric energy meter based on computer vision

Abstract

The present invention relates to the field of electric energy meters, and more specifically, to an automatic meter reading method for electric energy meters based on computer vision. The method comprises: taking the same electric energy meter image and image label in history as samples to construct an electric energy meter sample set, wherein the image label comprises a candidate frame and an electric energy meter digital value; for any sample, taking two candidate frames in the same column as a frame pair, and calculating the difficulty of classifying the frame pair; constructing a loss function according to the difficulty of classifying for training a preset network, outputting the trained preset network with the electric energy meter image taken in real time, outputting the electric energy meter digital value, and completing automatic meter reading. Through the technical solution of the present invention, the high efficiency of electric energy meter reading can be guaranteed, while the meter reading accuracy can be improved.

Description

Technical Field

[0001] The present invention relates to the field of electric energy meters, and more specifically, to an automatic meter reading method for electric energy meters based on computer vision. Background Art

[0002] As the main tool for power companies to measure users' energy consumption, the accuracy and timeliness of electric energy meters are of great significance to the stability of the power system and the income of power companies. The traditional way of reading electric energy meters mainly relies on manual inspection and meter reading, which has a series of challenges and shortcomings. With the acceleration of urbanization and the development of smart grids, the efficiency and accuracy of traditional meter reading methods can no longer meet the needs of modern society for power management and services. Power companies need a more intelligent and automated meter reading system to improve work efficiency, reduce labor costs, reduce errors and omissions, and provide more accurate data.

[0003] The existing Chinese patent application document with publication number CN104702920A discloses a method for automatic meter reading of an electric energy meter based on CMOS image acquisition, the method comprising: photographing the meter frame of the electric energy meter by means of a CMOS image sensor to obtain a meter frame image; performing image processing on the meter frame image obtained in step 1 by means of a digital signal processor to obtain an electric energy meter reading; and storing and displaying the electric energy meter reading obtained in step 2 by means of a FLASH memory and an LCD display screen, respectively.

[0004] However, the reading of the electric energy meter may rotate during operation, and the method in the above application document cannot accurately obtain the reading of the electric energy meter, resulting in inaccurate reading results of the electric energy meter. Summary of the invention

[0005] In order to solve the problem of inaccurate meter reading results of electric energy meters, the present invention proposes an automatic meter reading method of electric energy meters based on computer vision.

[0006] The present invention discloses an automatic meter reading method for electric energy meters based on computer vision, comprising: taking the same electric energy meter image and image label in history as samples to construct an electric energy meter sample set, wherein the image label comprises a candidate frame and an electric energy meter digital value; for any sample, taking two candidate frames in the same column as a frame pair, and calculating the difficulty of classifying the frame pair; constructing a loss function according to the difficulty of classifying for training a preset network, outputting the trained preset network with the electric energy meter image taken in real time, outputting the electric energy meter digital value, and completing automatic meter reading; wherein the loss function satisfies the relation: , Indicates frame pair The loss value, Indicates frame pair The difficulty of classification, represents the predicted value, represents the true value, represents a constant, Represents a logarithmic function.

[0007] By constructing a sample set of electric energy meter images and using the difficulty of box pair classification to design the loss function, the accuracy and robustness of the electric energy meter automatic meter reading system are effectively improved. Using two candidate boxes in the same column as box pairs, their difficulty of classification is calculated, which reflects the difficulty of identifying different areas in the electric energy meter image, especially for complex or fuzzy areas.

[0008] Preferably, the method further includes: selecting a candidate frame to be studied, wherein the number of columns of the candidate frame in the candidate frame to be studied is the same as the number of columns of the electric energy meter digits.

[0009] It can avoid the influence of interfering numbers on the prediction results and obtain more accurate meter reading results.

[0010] Preferably, obtaining the degree of difficulty in classification includes: using the result of negative correlation mapping of the absolute difference between the areas of two candidate boxes as the degree of difficulty in classification.

[0011] By calculating the absolute difference between the areas of two candidate boxes and using the result of negative correlation mapping as the degree of difficulty in classification, the difference between the areas in the energy meter image is effectively quantified. The blur or distortion caused by shooting angle, lighting, stains or image quality in the digital or pointer display part of the energy meter is reduced.

[0012] Preferably, obtaining the degree of difficulty in classification also satisfies the relationship:

[0013] , Indicates frame pair The difficulty of classification, Indicates frame pair The height of the first candidate box in Indicates frame pair The height of the second candidate box in represents the exponential function, represents the two-norm.

[0014] By calculating the binary norm of the height difference between two candidate boxes and further using an exponential function for weighting, the candidate boxes with larger differences have higher difficulty in classification, which has a more significant impact on model training.

[0015] Preferably, the preset network is a yolov5 model, the input end is used to receive the input of the electric energy meter image, the visual features are extracted through BackBone, the Neck receives the visual information and performs integrated analysis, and the digital value of the electric energy meter is output through the Head output layer.

[0016] Preferably, the training process of the preset network includes: using the images of electric energy meters in history as input information, and using the true values ​​of the digital values ​​of the electric energy meters in history as labels to obtain a set of training data; inputting the training data into the preset network to obtain output results; calculating the loss value of the output results and the labels through a loss function, back-propagating the error signal according to the loss value, updating the network parameters of the preset network to reduce the loss value; iteratively updating the network parameters of the preset network, and stopping the update when the preset network reaches the set maximum number of training times or the loss value is less than the set loss value to obtain a trained preset network.

[0017] Preferably, the outputting of the electric energy meter digital value comprises: the outputting of the electric energy meter digital value is the probability of different numbers appearing in each frame in the frame pair, and calculating the probability value of the digital pair according to the sum of the probabilities of two adjacent numbers to obtain the electric energy meter digital value.

[0018] Preferably, the probability value of the number pair satisfies the relationship:

[0019] , represents the probability value of a number pair, Indicates that the number value in the upper candidate box is The probability of Indicates the remainder, , indicating that the number value in the next candidate box is probability.

[0020] It reflects the continuous change pattern that may occur in the electricity meter numbers during the actual shooting process. Especially under complex image conditions, the correlation between numbers will help reduce the probability of misidentification, thereby improving the accuracy and stability of the system.

[0021] Beneficial effects of the present invention:

[0022] The present invention combines historical electricity meter images with label data and uses a preset network to automatically identify and read the electricity meter images. By calculating the difficulty of classifying the candidate frame, an adaptive loss function is designed to guide the network to learn more accurate digital values ​​of the electricity meter. Not only is the accuracy of automatic recognition of the electricity meter improved, but it can also be dynamically adjusted for difficult areas in different electricity meter images, such as blur, overlap, and occlusion. Through accurate modeling of candidate frames and reasonable estimation of digital pair probabilities, errors are effectively reduced and the ability to recognize numbers in electricity meter images is enhanced. During the training process, the network parameters are optimized through back propagation, so that the model can automatically adapt to different shooting conditions and lighting changes according to the characteristics of various electricity meter images, thereby improving the accuracy and efficiency of electricity meter reading in practical applications.

[0023] The present invention can reduce the possibility of human error, greatly reduce the cost of manual meter reading, and improve the accuracy and efficiency of electric energy meter reading recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] By reading the following detailed description with reference to the accompanying drawings, the above and other objects, features and advantages of the exemplary embodiments of the present invention will become readily understood. In the accompanying drawings, several embodiments of the present invention are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein:

[0025] Figure 1 The present invention is a flowchart of an automatic meter reading method for an electric energy meter based on computer vision. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0027] It should be understood that when the terms "first", "second", etc. are used in the claims, descriptions, and drawings of the present invention, they are only used to distinguish different objects, rather than to describe a specific order. The terms "include" and "comprise" used in the description and claims of the present invention indicate the presence of the described features, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their collections.

[0028] The present invention provides an automatic meter reading method for electric energy meters based on computer vision. Figure 1 As shown, the automatic meter reading method of electric energy meter based on computer vision includes steps S1 to S3, which are described in detail below.

[0029] S1, taking the same electric energy meter image and image label in history as samples to construct an electric energy meter sample set, the image label includes a candidate box and an electric energy meter digital value.

[0030] In one embodiment, in a modern intelligent power system, automatic meter reading is one of the important means to improve work efficiency and reduce labor costs.

[0031] Obtaining an image of an electric energy meter taken with a CCD camera in the past is a common method of this technology. However, since the electric energy meter may be affected by many factors in the actual working environment, such as shooting angle, lighting conditions, reflection, and occlusion of the meter reading position, the captured image can often only show a partial or blurred image of the number. In this case, the traditional digital recognition model often cannot achieve a high accuracy rate.

[0032] In order to solve this problem, the present invention uses the same electric energy meter image and image label as a sample, and the image label includes a candidate frame and an electric energy meter digital value.

[0033] Before determining the candidate boxes, they need to be screened first, because there may be interference numbers on the electric energy meter dial, which are generally household numbers or electric energy meter models, so a row of candidate boxes with the same number of columns as the electric energy meter numbers is selected.

[0034] Exemplarily, the electric energy meter dial has five columns of numbers, and a row of candidate boxes with five columns is selected as the candidate box to be studied.

[0035] S2: For any sample, the two candidate boxes in the same column are regarded as box pairs, and the difficulty of classification of the box pairs is calculated.

[0036] It should be noted that in the images of the electric energy meters taken by the camera, there may be incomplete numbers, which will bring significant challenges to the number recognition process. The incomplete numbers not only lack visual completeness, but the missing parts greatly reduce the characteristic information of the numbers, which leads to a sharp increase in the difficulty of number classification and recognition. As the degree of incompleteness increases, the difficulty of number recognition increases exponentially, especially when the incomplete area is large, even humans find it difficult to accurately distinguish the specific value of the number. In this case, traditional number recognition models, especially those based on deep learning, often fail to achieve the ideal accuracy.

[0037] However, in the actual meter reading process, although the incomplete numbers will affect the accuracy of recognition, due to the special design of the meter dial, the incomplete numbers do not disappear completely randomly. The meter usually records the electric energy by rotating the disk, which makes the numbers on the dial not disappear or be incomplete with equal probability during a rotation, but are affected by the position and angle. For example, when a part of a number is missing, the missing degree of another number is usually smaller. This phenomenon is due to the physical characteristics of the meter dial and the reading method.

[0038] For example, if the number 3 is more incomplete in the image, it can be inferred that the number 4 that follows it is usually relatively more complete. This is because the numbers on the meter are arranged in order, the display of each number is closely related to the rotation position of the dial, and the visible part of each number will have a certain relative position relationship during the rotation process. Therefore, when a certain number is partially blocked or blurred due to the influence of the rotation angle, the appearance of another number is often less affected, which may present a clearer image.

[0039] Therefore, the two candidate boxes in the same column are taken as box pairs, and the difficulty of classification of the box pairs is calculated. The maximum difficulty of classification is when both the upper and lower numbers are incomplete in half, that is, when the difference in the incomplete area of ​​the upper and lower numbers is the smallest, the difficulty of classification is the maximum.

[0040] In one embodiment, the degree of difficulty in classification includes: using the result of negative correlation mapping of the absolute difference between the areas of two candidate boxes as the degree of difficulty in classification.

[0041] The formula is: , Indicates frame pair The difficulty of classification, Indicates frame pair The height of the first candidate box in Indicates frame pair The width of the first candidate box in , Indicates frame pair The height of the second candidate box in Indicates frame pair The width of the second candidate box in , Represents an exponential function.

[0042] In another embodiment, the degree of difficulty in classification further satisfies the relationship:

[0043] , Indicates frame pair The difficulty of classification, Indicates frame pair The height of the first candidate box in Indicates frame pair The height of the second candidate box in represents the exponential function, represents the two-norm.

[0044] By calculating the binary norm of the height difference between two candidate boxes and further weighting them using an exponential function, the candidate boxes with larger differences have higher difficulty in classification, which has a more significant impact on model training. In practical applications, the numbers in meter images are often incomplete or partially occluded. The calculation of the difficulty in classification can help the model identify which areas have a greater impact on the classification results and adjust the learning strategy accordingly. Specifically, a higher difficulty in classification indicates that the model needs to process these partially occluded areas more finely, enabling the model to maintain high accuracy when faced with partially missing or incomplete images.

[0045] S3, constructs a loss function based on the degree of difficulty in classification for training the preset network, outputs the real-time captured electric energy meter image to the trained preset network, outputs the digital value of the electric energy meter, and completes automatic meter reading.

[0046] In one embodiment, the preset network is a yolov5 model, the input end is used to receive the input of the electric energy meter image, the visual features are extracted through BackBone, the Neck receives the visual information and performs integrated analysis, and the digital value of the electric energy meter is output through the Head output layer.

[0047] The training process of the preset network includes: taking the images of electric energy meters in history as input information, and taking the real values ​​of the digital values ​​of the electric energy meters in history as labels, to obtain a set of training data; inputting the training data into the preset network to obtain output results; calculating the loss values ​​of the output results and labels through the loss function, back-propagating the error signal according to the loss value, updating the network parameters of the preset network to reduce the loss value; iteratively updating the network parameters of the preset network, and stopping the update when the preset network reaches the set maximum number of training times or the loss value is less than the set loss value to obtain a trained preset network.

[0048] Exemplarily, updating is stopped when the preset network training times reaches 200 times or the loss value is less than 0.0001.

[0049] Among them, the loss function satisfies the relationship:

[0050] , Indicates frame pair The loss value, Indicates frame pair The difficulty of classification, represents the predicted value, represents the true value, represents a constant, Represents a logarithmic function.

[0051] The real-time captured energy meter image is output to the trained preset network to output the digital value of the energy meter.

[0052] Specifically, when using the trained preset network to recognize the electric energy meter image taken in real time, the preset network will output the positions of all numbers in the electric energy meter image and the possible numbers at that position. The position includes the coordinates of the candidate box, the width of the candidate box, the height of the candidate box, and the probability of outputting the number 0 at that position, the probability of the number 1, the probability of the number 2, the probability of the number 3, the probability of the number 4, the probability of the number 5, the probability of the number 6, the probability of the number 7, the probability of the number 8 and the probability of the number 9.

[0053] The output candidate boxes are arranged according to the column coordinates to obtain the final electric energy meter reading. When a column detects a box pair, the probability value of the number pair is calculated according to the probability of different numbers appearing in each box in the box pair. The probability value of the number pair satisfies the relationship:

[0054] , represents the probability value of a number pair, Indicates that the number value in the upper candidate box is The probability of Indicates the remainder, , indicating that the number value in the next candidate box is probability.

[0055] The remainder operation can loop back from the number 9 to the number 0.

[0056] It should be noted that due to the unique structure of the electricity meter dial, the numbers are adjacent to each other. Therefore, when a frame pair appears, if the probability of number 5 in the upper candidate frame of the frame pair is the highest, then the probability value of the number in the lower candidate frame can only be calculated using the probability of number 6.

[0057] From the number 0 to the number 9 , and obtain the probability values ​​of all digital pairs. The case with the largest probability value of the digital pair is the reading of the electric energy meter.

[0058] At this point, the automatic meter reading of the electricity meter is completed.

[0059] Although this specification has shown and described a number of embodiments of the present invention, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Those skilled in the art will conceive of many modifications, changes and alternatives without departing from the ideas and spirit of the present invention. It should be understood that in the practice of the present invention, various alternatives to the embodiments of the present invention described herein may be employed.

[0060] The above are all preferred embodiments of the present invention, and are not intended to limit the protection scope of the present invention. Therefore, any equivalent changes made based on the structure, shape, and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. The automatic meter reading method of electric energy meter based on computer vision is characterized in that: include: The same electric energy meter image and image label in history are used as samples to construct an electric energy meter sample set, where the image label includes a candidate box and an electric energy meter digital value; For any sample, take the two candidate boxes in the same column as a box pair and calculate the difficulty of classification of the box pair; A loss function is constructed based on the degree of difficulty in classification to train the preset network. The real-time image of the electric energy meter is input into the trained preset network to output the digital value of the electric energy meter to complete automatic meter reading. Among them, the loss function satisfies the relationship: , Indicates frame pair The loss value, Indicates frame pair The difficulty of classification, represents the predicted value, represents the true value, represents a constant, represents the logarithmic function; Obtaining the degree of difficulty in classification also satisfies the relationship: , Indicates frame pair The difficulty of classification, Indicates frame pair The height of the first candidate box in Indicates frame pair The height of the second candidate box in represents the exponential function, represents the two-norm; The preset network is a yolov5 model, the input end is used to receive the input of the electric energy meter image, extract visual features through BackBone, receive visual information through Neck and perform integrated analysis, and output the digital value of the electric energy meter through the Head output layer; The training process of the preset network includes: The images of electric energy meters in history are used as input information, and the real values ​​of the digital values ​​of electric energy meters in history are used as labels to obtain a set of training data; Input the training data into the preset network to obtain the output result; The loss function is used to calculate the loss value of the output result and the label, and the error signal is back-propagated according to the loss value to update the network parameters of the preset network to reduce the loss value. Iteratively update the network parameters of the preset network. When the preset network reaches the set maximum number of training times or the loss value is less than the set loss value, stop updating to obtain a trained preset network.

2. The automatic meter reading method of electric energy meter based on computer vision according to claim 1 is characterized in that: Also includes: Select the candidate frame to be studied, wherein the number of columns of the candidate frame in the candidate frame to be studied is the same as the number of columns of the electric energy meter digits.

3. The automatic meter reading method of electric energy meter based on computer vision according to claim 1 is characterized in that: The output electric energy meter digital value includes: The output digital value of the electric energy meter is the probability of different numbers appearing in each box in the box pair. The probability value of the number pair is calculated according to the sum of the probabilities of two adjacent numbers to obtain the digital value of the electric energy meter.

4. The automatic meter reading method of electric energy meter based on computer vision according to claim 3 is characterized in that: The probability value of the number pair satisfies the relationship: , represents the probability value of a number pair, Indicates that the number value in the upper candidate box is The probability of Indicates the remainder, , indicating that the number value in the next candidate box is probability.

Images