An image recognition-based household meter state checking method

Abstract

The application relates to the technical field of image recognition processing, and discloses a household electric meter state checking method based on image recognition, which comprises the following steps: obtaining a target picture containing an electric meter image; performing electric meter positioning on the target picture to obtain electric meter image information; performing electric meter component positioning and damage state recognition according to the electric meter image information; obtaining state information of corresponding components according to the results of the electric meter component positioning; and obtaining the position and color information of electric meter electric wires according to the electric meter image information. The related data information of multiple electric meters can be obtained through one-time shooting, and the efficiency and accuracy of electric meter state and information checking are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image recognition processing, in particular to a household electric meter state checking method based on image recognition. BACKGROUND

[0002] The smart electric meter, as a power supply company asset, is used for user electricity consumption metering device, whether it is normal operation is the responsibility and obligation that the power supply company must guarantee. At present, the state grid requires that the metering device of the on-site power supply company asset is checked once a year, abnormal metering devices are found in time, replaced in time, and systematically registered and archived to avoid customer complaint events caused by abnormal metering devices. However, this work requires high professional requirements, and ordinary area managers or meter reading and fee collection personnel and ordinary collection and operation and maintenance personnel are difficult to effectively and comprehensively check related abnormal hidden dangers after reaching the scene, and the safety hidden danger, customer dispute or economic loss of the power supply company is more likely to occur. In addition, the work is huge, the traditional manual checking has poor input-output benefit, and at present, this work is difficult to improve the quality of this business due to the lack of effective intelligent means. SUMMARY

[0003] The purpose of the present application is to provide a household electric meter state checking method based on image recognition, which solves the following technical problems:

[0004] 1. The metering box device checking requires high professional requirements, and ordinary personnel are difficult to master;

[0005] 2. The skills of the on-site checking personnel are uneven, and the checking quality is difficult to control;

[0006] 3. The checking technology points are easy to overlook;

[0007] 4. After the problem checking is found, the defect elimination and closed loop management is difficult.

[0008] The purpose of the present application can be achieved by the following technical solutions:

[0009] A household electric meter state checking method based on image recognition, comprising:

[0010] Obtaining a target picture containing an electric meter image;

[0011] Performing electric meter positioning on the target picture to obtain electric meter image information;

[0012] According to the electric meter image information, electric meter component positioning and damage state recognition are performed respectively;

[0013] According to the results of the electric meter component positioning, the state information of the corresponding components is obtained;

[0014] According to the meter image information, position and color information of a meter wire are acquired;

[0015] The meter component positioning is configured to acquire screen display information of the meter and other component position information, and the damage state recognition is configured to determine whether the meter is in a damage state.

[0016] According to the technical solution, a target picture is obtained by photographing the meter, and then the meter and its components in the target picture are automatically classified and positioned, and a corresponding position picture is extracted, which assists a user in intuitively understanding the structure of the meter and determining whether the meter is in a damage state at the first time. Therefore, the target picture may include more than one meter, and therefore, the related data information of multiple meters can be obtained by one-time shooting, thereby improving the efficiency and accuracy of meter state and information checking.

[0017] As a further scheme of the application, the meter positioning comprises:

[0018] The target picture is input into a trained meter recognition model, and the meter recognition model outputs a meter positioning result.

[0019] The meter recognition model is a model obtained by training a data set using a target detection technology of a YOLO series.

[0020] Before the meter recognition model is trained, a data set of a three-phase meter needs to be prepared, and the area where the meter is located is labeled for each picture containing the meter in the data set. The data set is divided into a training set and a test set, and the data amount ratio of the training set to the test set is 7:3. Next, a target detection technology of a YOLO series is used to train the data set to obtain the meter recognition model, and the model is loaded to recognize the information about the meter in the input picture. The screen recognition model in the following is the same.

[0021] As a further scheme of the application, the obtaining of the meter image information comprises:

[0022] According to the acquired meter positioning result, a tensor matrix of the image of the target picture is cut to obtain a tensor matrix about the meter and saved as the meter image information.

[0023] The process can be represented as:

[0024]

[0025] In the formula, Pic is a tensor matrix of the target picture, zx, zy, yx and yy are the meter positioning results, respectively, coordinates of the meter in the target picture, and Meter represents a tensor matrix of the meter image information.

[0026] As a further scheme of the present application: the acquisition method of the screen position information of the electric meter is:

[0027] inputting the electric meter image information into the trained screen recognition model, and the screen recognition model outputs a screen positioning result;

[0028] the screen recognition model is a model obtained by training a data set using a target detection technology of a YOLO series;

[0029] by the obtained screen positioning result, the tensor matrix of the electric meter image information is cut to obtain a tensor matrix of the electric meter screen and saved as a picture;

[0030] The process can be represented as:

[0031]

[0032] wherein, Meter is the tensor matrix of the electric meter image information, sx, sy, xx and xy are the screen positioning results, respectively the coordinates of the screen in the electric meter image information, and Ping represents the tensor matrix of the electric meter screen.

[0033] As a further scheme of the present application: the acquisition method of the screen display information includes:

[0034] using a trained classification model to judge the state of the screen using image classification technology, first, a data set is made, two folders train and valid are created to represent the training set and the validation set, each folder contains four folders normal, crash, powerless and undervoltage to represent the four categories of the electric meter screen. The data set is trained using resnet50. According to the data set, an image classification model is trained, which can classify the electric meter screen.

[0035] convert the tensor matrix of the electric meter screen into a screen picture in the form of image, and then pre-process the screen picture;

[0036] the pre-processing operation includes Resize operation, CenterCrop operation and Normalize normalization operation;

[0037] the Resize operation:

[0038] adjust the short side of the input screen image to a preset length size, and adjust the long side according to the corresponding proportion;

[0039] Let the high and wide of the input screen image be represented as (h, w), where the wide w is the short side, then this process can be represented as:

[0040] (h1, w1) = (size x h ÷ w, size)

[0041] Where (h1, w1) represents the size of the screen picture after the Resize operation;

[0042] The Center Crop operation:

[0043] Adjust the size of the image to (size, size) according to the center position of the screen image;

[0044] The Normalize normalization operation:

[0045] Since the screen image is a three-channel image, its mean and standard deviation are set to [0.485, 0.456, 0.406] and [0.229, 0.224, 0.225] respectively;

[0046] This calculation process can be represented as:

[0047]

[0048] Where input and output represent the input image and the output image respectively, mean and std are the mean and the standard deviation respectively.

[0049] As a further scheme of the application: the method for obtaining the damage state comprises:

[0050] The meter image information is input into the trained state judgment model, and the state judgment model outputs a damage state judgment result. Whether the meter is damaged is a binary classification problem, which is divided into normal and damage classes, so when the data set is made, the training set and the test set also only contain the corresponding two files.

[0051] As a further scheme of the application: the method for obtaining the position and color information of the meter wire comprises:

[0052] Step S61, collect all colors of the wires of various types of meters, determine the RGB value range of all colors, then identify the meter wire, and obtain the wire image;

[0053] Step S62, the acquired wire image is converted from the RGB color space to the HSV color space, the mask of the preset color is obtained by making the image values not belonging to the preset color range to 0, and the pixel values of the current pixel point and the surrounding adjacent pixel points are obtained by median filtering, the pixel values are sorted, and the pixel value located in the middle position is taken as the pixel value of the current pixel point;

[0054] Step S63, the mask of the preset color generated in the step S62 is traversed, the contour area of the mask is detected and calculated, and the sum of the contour areas of all the masks is taken as the area occupied by the preset color in the image;

[0055] Step S64, the areas of various colors corresponding to the wire image in the image are obtained, the areas of the colors other than black are multiplied by 10, and the obtained results are the final area values of the colors; step S65, the color corresponding to the maximum value of the area values of all colors is the main color of the input image;

[0056] Step S66, the coordinates of the wire in the wire image are obtained, the coordinates of the upper left corner of the recognized wire are sorted by bubble sorting, the colors of the corresponding wires are sequentially output from small to large according to the coordinates, and the type of the electric meter is judged according to the color sorting of the electric meter wire.

[0057] As a further scheme of the present application, the other components include an indicator light and a bar code.

[0058] The coordinates of the electric meter, the screen of the electric meter, the indicator light, the bar code and the wire connection are obtained, and a frame is drawn and labeled with text at the corresponding coordinates on the target picture.

[0059] The present application has the following advantages: the electric meter is photographed to obtain a target picture, then the electric meter and its components in the target picture are automatically classified and positioned, and the corresponding position picture is extracted, which helps the user to intuitively understand the structure of the electric meter and judge whether the electric meter is in a damaged state at the first time; therefore, the target picture may include not only one electric meter, so that the related data information of multiple electric meters can be obtained by one shooting, and the efficiency and accuracy of the electric meter state and information checking are improved. BRIEF DESCRIPTION OF DRAWINGS

[0060] The present application will be further described below with reference to the accompanying drawings.

[0061] Figure 1 The present application is a schematic flowchart of the principle of the household electric meter state checking method.

[0062] Figure 2 The present application is a specific flowchart of the household electric meter state checking method. DETAILED DESCRIPTION

[0063] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0064] Please refer to Figure 1 and Figure 2 The present application is a household meter state checking method based on image recognition, which comprises:

[0065] obtaining a target picture containing a meter image;

[0066] performing meter positioning on the target picture to obtain meter image information;

[0067] performing meter component positioning and damage state recognition according to the meter image information;

[0068] obtaining state information of corresponding components according to the results of the meter component positioning;

[0069] obtaining position and color information of meter wires according to the meter image information;

[0070] The meter component positioning is used to obtain screen display information and other component position information of the meter, and the damage state recognition is used to determine whether the meter is in a damaged state.

[0071] Through the above technical solution, a target picture is obtained by photographing a meter, and then the meter and its components in the target picture are automatically classified and positioned, and corresponding position pictures are extracted, which helps users to intuitively understand the structure of the meter and obtain a judgment on whether the meter is in a damaged state at the first time. Therefore, the target picture may include not only one meter, so that related data information of multiple meters can be obtained by one photographing, thereby improving the efficiency and accuracy of meter state and information checking.

[0072] As a further scheme of the present application, the meter positioning comprises:

[0073] inputting the target picture into a trained meter recognition model, and outputting meter positioning results by the meter recognition model;

[0074] The meter recognition model is a model obtained by training a data set using a target detection technology of a YOLO series.

[0075] As a further scheme of the present application, the obtaining of the meter image information comprises:

[0076] According to the obtained meter positioning result, a tensor matrix of an image of the target picture is cut to obtain a tensor matrix related to the meter and saved as meter image information;

[0077] The process can be represented as:

[0078]

[0079] Wherein, Pic is the tensor matrix of the target picture, zx, zy, yx and yy are the meter positioning results, respectively the coordinates of the meter in the target picture, and Meter represents the tensor matrix of the meter image information.

[0080] As a further scheme of the application: the meter screen position information acquisition method comprises:

[0081] The meter image information is input into a trained screen recognition model, and the screen recognition model outputs a screen positioning result;

[0082] The screen recognition model is a model obtained by training a data set using a YOLO series target detection technology;

[0083] According to the obtained screen positioning result, a tensor matrix of the meter image information is cut to obtain a tensor matrix related to the meter screen and saved as a picture;

[0084] The process can be represented as:

[0085]

[0086] Wherein, Meter is the tensor matrix of the meter image information, sx, sy, xx and xy are the screen positioning results, respectively the coordinates of the screen in the meter image information, and Ping represents the tensor matrix of the meter screen.

[0087] As a further scheme of the application: the screen display information acquisition method comprises:

[0088] A trained classification model is called using resnet50 to classify the image of the meter screen;

[0089] The tensor matrix of the meter screen is converted into an image form of the screen picture, and the screen picture is preprocessed;

[0090] The preprocessing operation comprises Resize operation, CenterCrop operation and Normalize normalization operation.

[0091] The Resize operation:

[0092] Adjust the short side of the input screen image to a preset length size, and adjust the long side according to the corresponding proportion;

[0093] Let the height and width of the input screen image be (h, w), where the width w is the short side, then this process can be represented as:

[0094] (h1, w1) = (size x h ÷ w, size)

[0095] Where (h1, w1) represents the size of the screen picture after the Resize operation;

[0096] The Center Crop operation:

[0097] Adjust the size of the image to (size, size) according to the center position of the screen image;

[0098] The Normalize normalization operation:

[0099] Since the screen image is a three-channel image, its mean and standard deviation are set to [0.485, 0.456, 0.406] and [0.229, 0.224, 0.225], respectively;

[0100] This calculation process can be represented as:

[0101]

[0102] Where input and output represent the input image and the output image, respectively, and mean and std represent the mean and the standard deviation, respectively.

[0103] As a further scheme of the present application: the method for obtaining the damage state comprises:

[0104] Inputting the meter image information into the trained state judgment model, the state judgment model outputs the damage state judgment result.

[0105] As a further scheme of the present application: the method for obtaining the position and color information of the meter wire comprises:

[0106] Step S61, collect all colors of the wires of various types of meters, determine the RGB value range of all colors, then identify the meter wire and obtain the wire image; for example, the RGB range of red is from [156, 43, 46] to [180, 255, 255], and when the RGB of a certain region of the image is in this range, this region is considered to be red.

[0107] Step S62, the acquired wire image is converted from the RGB color space to the HSV color space, the mask of the preset color is obtained by making the image values not belonging to the preset color range to 0, and the pixel values of the current pixel point and the surrounding adjacent pixel points are obtained by median filtering, the pixel values are sorted, and the pixel value located in the middle position is taken as the pixel value of the current pixel point;

[0108] Step S63, the mask of the preset color generated in the step S62 is traversed, the contour area of the mask is detected and calculated, and the sum of the contour areas of all the masks is taken as the area occupied by the preset color in the image;

[0109] Step S64, the areas of various colors corresponding to the wire image in the image are obtained, the areas of the colors other than black are multiplied by 10, and the obtained results are the final area values of the colors; the purpose of this step is to sort the influence of the interference color black, and therefore compensation is added to other colors.

[0110] Step S65, the color corresponding to the maximum value of the area values of all colors is the main color of the input image; for example, the input image is the image of a red electric meter wire, then the area value of red is the largest after color detection, and the output result is red, that is, the recognized wire color.

[0111] Step S66, the coordinates of the wire in the wire image are obtained, the coordinates of the upper left corner of the recognized wire are sorted by bubble sorting, and the colors of the corresponding wires are sequentially output from small to large according to the coordinates; the type of the electric meter is determined according to the color sorting of the electric meter wire.

[0112] As a further scheme of the application, the other components include an indicator light and a bar code.

[0113] The coordinates of the electric meter, the screen of the electric meter, the indicator light, the bar code and the wire connection are obtained, and a frame is drawn and labeled with text at the corresponding coordinates on the target picture.

[0114] The above describes one embodiment of the application in detail, but the content is only the preferred embodiment of the application, and cannot be considered as limiting the scope of the application. Any equivalent changes and improvements made according to the scope of the application should still belong to the patent coverage of the application.

Claims

1. An image recognition-based method for checking the status of a domestic meter, characterized in that, The method comprises the following steps: acquiring a target picture containing an electric meter image; performing electric meter positioning on the target picture to obtain electric meter image information; performing electric meter component positioning and damage state identification according to the electric meter image information respectively; acquiring state information of corresponding components according to the results of electric meter component positioning; acquiring position and color information of electric meter electric wires according to the electric meter image information; wherein the electric meter component positioning is used to acquire screen display information and other component position information of the electric meter, and the damage state identification is used to determine whether the electric meter is in a damaged state; the electric meter positioning comprises the following steps: inputting the target picture into a trained electric meter identification model, and outputting electric meter positioning results by the electric meter identification model; the electric meter identification model is a model obtained by training a data set using a target detection technology of a YOLO series; the electric meter image information acquisition comprises the following steps: cutting a tensor matrix of an image of the target picture according to the acquired electric meter positioning results to obtain a tensor matrix related to the electric meter and save it as the electric meter image information; the process can be represented as: ； wherein Pic is the tensor matrix of the target picture, zx, zy, yx and yy are the electric meter positioning results, which are respectively coordinates of the electric meter in the target picture, and Meter represents the tensor matrix of the electric meter image information.

2. The image recognition based utility meter status verification method of claim 1, wherein, the screen position information acquisition of the electric meter comprises the following steps: inputting the electric meter image information into a trained screen identification model, and outputting screen positioning results by the screen identification model; the screen identification model is a model obtained by training a data set using a target detection technology of a YOLO series; cutting a tensor matrix of the electric meter image information according to the acquired screen positioning results to obtain a tensor matrix related to the electric meter screen and save it as a picture; the process can be represented as: ； wherein Meter is the tensor matrix of the electric meter image information, sx, sy, xx and xy are the screen positioning results, which are respectively coordinates of the screen in the electric meter image information, and Ping represents the tensor matrix of the electric meter screen.

3. The image recognition based utility meter status verification method of claim 2, wherein, the screen display information acquisition method comprises the following steps: using a trained classification model to classify the image of the electric meter screen by using resnet50; converting the tensor matrix of the electric meter screen into an image form of a screen picture, and then pre-processing the screen picture; the pre-processing operation comprises Resize operation, CenterCrop operation and Normalize normalization operation; the Resize operation comprises the following steps: adjusting the short side of the input screen image to a preset length size, and adjusting the long side according to the corresponding proportion; letting the height and width of the input screen image be represented as (h, w), wherein the width w is the short side, and the process can be represented as: ； wherein (h1, w1) represents the size of the screen picture after the Resize operation; the CenterCrop operation comprises the following steps: Adjust the size of the image to (size, size) according to the center position of the screen image; The Normalize normalization operation: Since the screen image is a three-channel image, the mean and standard deviation are set to [0.485, 0.456, 0.406] and [0.229, 0.224, 0.225] respectively; This calculation process can be represented as: ； Wherein, input and output respectively represent the input image and the output image, and mean and std respectively represent the mean and the standard deviation.

4. The image recognition based utility meter status verification method of claim 1, wherein, The method for obtaining the damage state comprises: Inputting the meter image information into the trained state judgment model, and outputting a damage state judgment result by the state judgment model.

5. The image recognition based utility meter status verification method of claim 1, wherein, The method for obtaining the position and color information of the meter wire comprises: Step S61, collecting all colors of the wires of various types of meters, and determining the RGB value range of all colors; Step S62, converting the obtained wire image from the RGB color space to the HSV color space, obtaining a mask of the preset color by changing the image values not belonging to the preset color range to 0, and then obtaining the pixel values of the current pixel point and the surrounding adjacent pixel points by median filtering, sorting these pixel values, and taking the pixel value located in the middle position as the pixel value of the current pixel point; Step S63, traversing the mask of the preset color generated in step S62, detecting and calculating the contour area of the mask, and taking the sum of the contour areas of all masks as the area occupied by the preset color in the image; Step S64, obtaining the areas of various colors in the image corresponding to the wire image, multiplying the areas of the colors other than black by 10, and obtaining the final area values of these colors; Step S65, taking the color corresponding to the maximum area value of all colors as the main color of the input image; Step S66, obtaining the coordinates of the wire in the wire image, performing bubble sorting according to the coordinates of the top left corner of the recognized wire, and sequentially outputting the colors of the corresponding wires according to the coordinates from small to large; and judging the type of the meter according to the color sorting of the meter wire.

6. The image recognition based utility meter status verification method of claim 1, wherein, The other components include an indicator light and a barcode; Obtaining the coordinates of the meter, the meter screen, the indicator light, the barcode and the wire connection, and then drawing a frame at the corresponding coordinates on the target picture and labeling with text.

Images