A method for reading oil meters in substations based on deep learning

Abstract

The present invention relates to a substation oil meter reading method based on deep learning, which uses a deep learning target detection method to locate the position of the oil meter. Compared with a template matching method, it is more robust and can accurately locate the position of the oil meter in various viewing angles and environments. A deep learning semantic method is used to perform perspective correction of the oil meter, which can improve the accuracy of the oil meter reading. The caliper position positioning based on image enhancement can cope with the interference of factors such as aging and scratches on the surface glass of the oil meter.

Description

Technical Field

[0001] The present invention belongs to the technical field of smart substations, and in particular relates to a substation oil meter reading method based on deep learning. Background Art

[0002] With the introduction of the concept of smart substations, more and more smart inspection robots are replacing staff to perform daily substation equipment inspections. In the prior art, the patent with publication number CN105608703A "A method for detecting the oil level of a current transformer by a smart patrol robot in a substation" involves a visual inspection method for the oil level of a current transformer in its overall process scheme. It uses template matching to locate the position of the current transformer and obtains the position and reading of the caliper by analyzing the edge information in the regional image.

[0003] Disadvantages of the existing technology: The positioning method based on template matching has poor robustness and is prone to errors when the target image is greatly different from the template, such as the influence of different acquisition angles and environmental factors; when the angle of the acquired image is greatly offset, the reading accuracy is easily affected if no image correction is performed; in the case of old oil meter equipment in the substation, aging and scratches on the glass surface, the caliper position cannot be read directly using the edge information method. Summary of the invention

[0004] In order to solve the above technical problems, the present invention provides a substation oil meter reading method based on deep learning.

[0005] The present invention is achieved through the following technical solutions:

[0006] A method for reading oil meters in substations based on deep learning, comprising the steps of:

[0007] 1.1 The mobile robot collects the oil meter image through the telephoto lens at the first predetermined position;

[0008] 1.2 Based on the deep learning target detection method, locate the oil meter in the image;

[0009] 1.3 Based on the deep learning semantic segmentation method, the oil meter image is corrected by perspective transformation;

[0010] 1.4 Based on the image enhancement processing method, the caliper position of the oil gauge is obtained;

[0011] 1.5 Calculate the oil gauge reading based on the upper and lower limit position relationship between the caliper and the oil gauge;

[0012] 1.6 Output the oil gauge reading.

[0013] Furthermore, in step 1.2, the target position of the oil meter within the image range is obtained through a convolutional neural network, and the target is marked out from the image with a bounding box.

[0014] Furthermore, the position of the fuel gauge is located through a convolutional neural network model, specifically, a detection frame of the target pair is obtained through classification recognition and regression calculation.

[0015] Furthermore, in step 1.3, the method based on deep learning semantic segmentation performs perspective transformation correction on the oil meter image, which can be divided into the following steps:

[0016] 3.1 Expand the target detection area to obtain the semantic segmentation processing area;

[0017] 3.2 Perform semantic segmentation on the target detection area to obtain segmentation masks of left and right straight lines and upper and lower arcs;

[0018] 3.3 Perform point extraction and line fitting on the left and right straight line segmentation masks to obtain left and right straight line expressions;

[0019] 3.4 Perform point extraction and binomial fitting on the upper and lower arc segmentation masks to obtain the quadratic expression of the upper and lower arcs;

[0020] 3.5 Calculate the intersection points P1 and P2 of the left straight line and the upper and lower arcs;

[0021] 3.5 Calculate the intersection points P3 and P4 of the right straight line and the upper and lower arcs;

[0022] 3.6 Use the intersection points P1, P2, P3, and P4 to perform perspective transformation and obtain the image after perspective transformation;

[0023] Furthermore, in step 3.3, after the left and right straight line points are extracted, the least squares method is used to fit them to obtain the left and right straight line expressions.

[0024] Further, in step 3.6, the perspective transformed image, i.e., the target rectangular area R ((rx1, ry1), (rx2, ry2)), is obtained through the intersection points P1, P2, P3, and P4, where (rx1, ry1) is the upper left corner coordinate of the target rectangular area, and (rx2, ry2) is the upper right corner coordinate of the target rectangular area, where:

[0025] rx1=min(xp1,xp2,xp3,xp4),

[0026] rx2=max(xp1,xp2,xp3,xp4),

[0027] ry1=min(yp1,yp2,yp3,yp4),

[0028] ry2=max(yp1,yp2,yp3,yp4),

[0029] Among them, the intersection points of the left straight line and the upper and lower arcs are P1(xp1, yp1) and P2(xp2, yp2), the intersection points of the right straight line and the upper and lower arcs are P3(xp3, yp3) and P4(xp4, yp4), min() and max() are minimum and maximum value operations respectively.

[0030] Furthermore, in step 1.4, a red channel image of the oil gauge image corrected in step 1.3 is obtained by channel separation, and Gaussian filtering is performed. The contrast of the caliper in the image is enhanced by equalization, and the mask of the caliper is obtained by image thresholding and mask optimization, and the area with the largest area is selected as the position of the caliper.

[0031] Furthermore, in step 1.5, after obtaining the caliper position, the ratio of the caliper height position to the oil gauge height is calculated to obtain the meter reading.

[0032] Compared with the prior art, the present invention has the following advantages:

[0033] 1. Use deep learning target detection to locate the fuel gauge. The target detection method is very robust and can accurately locate the fuel gauge in a variety of different perspectives and environments.

[0034] 2. Fuel gauge perspective correction based on deep learning semantics. The edge of the fuel gauge is divided into left and right straight lines and upper and lower arcs, and the perspective matrix is ​​obtained by combining semantic segmentation and fitting methods, which can achieve accurate fuel gauge perspective correction and improve the accuracy of fuel gauge readings.

[0035] 3. Caliper position positioning based on image enhancement. By using image enhancement and other image processing methods, the features of the image caliper in the image can be improved to cope with interference from factors such as aging and scratches on the surface glass of the oil meter. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The present invention will be further described in detail below with reference to the accompanying drawings;

[0037] Figure 1 It is the overall flow chart of the oil meter identification method of the substation;

[0038] Figure 2 It is the location map of the oil meter in the substation;

[0039] Figure 3.1 This is the expanded target detection area map;

[0040] Figure 3.2 It is the semantic segmentation map of the oil meter edge;

[0041] Figure 3.3 This is the oil meter edge fitting diagram;

[0042] Figure 3.4 Correct the image for oil gauge perspective;

[0043] Figure 4.1 It is the image inside the oil meter;

[0044] Figure 4.2 This is the red channel image of the oil meter;

[0045] Figure 4.3 Enhance the image for the red channel of the oil gauge;

[0046] Figure 4.4 Optimize mask for caliper;

[0047] Figure 4.5 This is the oil gauge reading diagram. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] The patrol robot performs patrol tasks in the substation. When it reaches the designated location, it uses a telephoto lens to collect images containing the oil meter. The area surrounded by the black rubber ring is the location of the oil meter, which needs to be located and identified. The robot will collect images at different angles, so it will obtain images of the oil meter at different perspective angles. When the perspective angle is large, if the reading is taken directly, there is a high probability of reading errors caused by the viewing angle. In addition, due to the use time and long-term exposure to the outside, the surface aging and scratches of the oil meter surface glass are also likely to interfere with the reading, and image enhancement and other methods are needed to optimize the processing.

[0050] The overall process of a method for reading oil meter in a substation is as follows: Figure 1 As shown, the process can be divided into the following steps:

[0051] 1.1 The mobile robot collects the oil meter image through a telephoto lens at a designated remote location;

[0052] 1.2 Based on the deep learning target detection method, locate the oil meter in the image;

[0053] 1.3 Based on the deep learning semantic segmentation method, the oil meter image is corrected by perspective transformation;

[0054] 1.4 Based on image processing methods such as image enhancement, the caliper position of the oil gauge is obtained;

[0055] 1.5 Calculate the oil gauge reading based on the upper and lower limit position relationship between the caliper and the oil gauge.

[0056] 1.6 Output the oil gauge reading.

[0057] In step 1.2, the location of the oil meter in the substation is:

[0058] The fuel gauge position positioning based on deep learning target detection is to obtain the target position of the fuel gauge within the image range through a convolutional neural network, and mark the target from the image with a bounding box. The process can be divided into training and reasoning stages. In the training stage, a large number of fuel gauge images under different perspectives and environments need to be collected for model training. In order to further enrich the training data and improve the robustness of the algorithm. The diversity of training data is further improved through data augmentation methods such as random brightness and random cropping. In the reasoning application stage, the trained model is used to locate the fuel gauge. Its essence is to obtain the detection frame of the target object through classification recognition and regression calculation. This patent mainly uses Yolov5 as the main framework for target detection, but the target detection method is not limited to FasterRcnn, Yolo and SSD series, etc. The results of its target detection are as follows Figure 2 Assume that the oil gauge detection frame obtained based on the deep learning target detection method is A(x1, y1, w2, h2), where (x1, y1) is the coordinate of the upper left corner of the detection frame A in the image, and w1, h1 are the width and height of the detection frame A respectively.

[0059] In step 1.3, the substation oil meter image is corrected

[0060] The patrol robot will collect images at different angles, so it will get images of the fuel gauge at different perspective angles. When the perspective angle is large, if the reading is taken directly, the perspective problem is likely to cause reading errors. Therefore, a method based on deep learning semantic segmentation is used to implement perspective correction of the fuel gauge. The correction process can be divided into the following steps:

[0061] 3.1 Expand the target detection area to obtain the semantic segmentation processing area

[0062] 3.2 Perform semantic segmentation on the target area to obtain segmentation masks of left and right straight lines and upper and lower arcs

[0063] 3.3 Perform point extraction and line fitting on the left and right straight line masks to obtain the left and right straight line expressions

[0064] 3.4 Perform point extraction and binomial fitting on the upper and lower arc masks to obtain the quadratic expression of the upper and lower arcs

[0065] 3.5 Calculate the intersection points P1 and P2 of the left straight line and the upper and lower arcs

[0066] 3.5 Calculate the intersection points P3 and P4 of the right straight line and the upper and lower arcs

[0067] 3.6 Use the intersection points P1, P2, P3, and P4 to perform perspective transformation and obtain the image after perspective transformation.

[0068] In step 3.1, expand the target detection area:

[0069] The essence of target detection is classification recognition and regression calculation. Sometimes the detection results obtained will have a certain deviation from the target position. In order to ensure that the entire oil gauge is within the target detection frame, the length and width of the detection frame need to be enlarged by a certain proportion. Set w_scale and h_scale as the horizontal and vertical offset magnification coefficients, respectively, and different sizes can be set, such as 0.1 and 0.1 respectively. Then, according to the detection frame A(x1, y1, w1, h1) and the offset coefficients w_scale and h_scale, the offset detection frame A2(x2, y2, w2, h2) is obtained, where x2 = xw*w_scale, y2 = yh*h_scale, w2 = w+2*w*w_scale, h2 = h+2*h*h_scale, and the expanded detection area is shown in 3.1.

[0070] In step 3.2, the oil meter edge semantic segmentation is performed:

[0071] It can be seen from the collected images that the overall appearance of the fuel gauge can be seen as consisting of two upper and lower semicircles and a rectangle. From different perspectives, the edge appearance can be seen as being connected by two upper and lower arcs and left and right straight lines. Therefore, the foreground categories of the semantic segmentation of the edge of the fuel gauge are the following four: left straight line, right straight line, upper arc and lower arc. Semantic segmentation is a basic task in computer vision. Its essence is to classify each pixel in the image. Therefore, the pixel at each position in the target image will be judged as one of the background, left straight line, right straight line, upper arc and lower arc. All pixels of the same category are called the foreground mask of the class. The method of semantic segmentation of the fuel gauge is also divided into training and inference stages. In the training stage, a large number of fuel gauge images under different perspectives and environmental factors are used for model training. In order to further improve the robustness of the algorithm, the diversity of training data is further enriched through data augmentation methods such as random brightness and random cropping. In the inference application stage, the trained model is used to perform semantic segmentation of the fuel gauge. The results of semantic segmentation are shown as follows. Figure 3.2 shown.

[0072] In steps 3.3-3.5, the oil gauge edge is fitted:

[0073] Before fitting the left and right straight lines, it is necessary to extract points from the foreground mask. The extraction method is to take the column coordinate average of all foreground masks in the same row as the column coordinate, and the row coordinate is the number of the row. After the left and right straight line points are extracted, the least squares method is used to obtain the linear expressions of the left and right straight lines. Mask point extraction is also performed on the upper and lower arc foreground masks. The extraction method is to take the row coordinate average of all foreground masks in the same column as the row coordinate, and the column coordinate is the number of the column. After the upper and lower arc points are extracted, quadratic terms are used to fit the extraction, respectively, as the quadratic expression of the upper and lower arcs. Then, the system of equations consisting of the linear expression of the left straight line and the quadratic expressions of the upper and lower arcs is solved to obtain the intersection points of the left straight line and the upper and lower arcs P1(xp1,yp1), P2(xp2,yp2). The system of equations consisting of the linear expression of the right straight line and the quadratic expressions of the upper and lower arcs is solved to obtain the intersection points of the right straight line and the upper and lower arcs P3(xp3,yp3), P4(xp4,yp4), and the results are as follows: Figure 3.3 shown.

[0074] In step 3.6, the oil gauge perspective correction:

[0075] The target rectangular area R ((rx1, ry1), (rx2, ry2)) after perspective transformation is obtained through the intersection points P1, P2, P3, and P4, where (rx1, ry1) is the upper left corner coordinate of the target rectangular area, and (rx2, ry2) is the upper right corner coordinate of the target rectangular area. Among them:

[0076] rx1=min(xp1,xp2,xp3,xp4),rx2=max(xp1,xp2,xp3,xp4),ry1=min(yp1,yp2,yp3,yp4),ry2=max(yp1,yp2,yp3,yp4),where min() and max() are the minimum and maximum operations respectively. The image after perspective transformation is as follows Figure 3.4 shown.

[0077] In step 1.4, the oil gauge caliper is positioned:

[0078] Through step 1.3, the corrected oil gauge image can be obtained, as shown in Figure 4.1 As shown in the figure. Due to the use time and long-term exposure to the outside, the aging and scratches on the surface of the oil gauge glass are easy to interfere with the reading, and it is necessary to optimize it by image enhancement and other methods. From the prior knowledge, we know that the color of the caliper is red. Therefore, the red channel image of the image is obtained by channel separation, and Gaussian filtering is performed to reduce the influence of surface scratches. The processed image is shown in the figure. Figure 4.2As shown in the figure. Due to reasons such as lighting and glass occlusion, the position of the caliper in the image is not obvious, and the contrast of the caliper in the image is further enhanced by equalization. The effect of the image enhancement is shown in the figure. Figure 4.3 As shown. And the caliper mask is obtained by image threshold and mask optimization in one step, as shown Figure 4.4 As shown, the area with the largest area is selected as the position of the caliper. After obtaining the caliper position, the ratio of the caliper height position to the oil gauge height is calculated to obtain the meter reading. The final reading result is as follows: Figure 4.5 shown.

[0079] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the steps of a substation oil meter reading method based on deep learning are implemented.

[0080] The present invention also provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of a substation oil meter reading method based on deep learning are implemented.

[0081] The specific embodiments described above further describe the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made without departing from the spirit and scope of the present invention also fall within the scope of protection of the present invention.

Claims

1. A method for reading oil meter in a substation based on deep learning, characterized in that it comprises the following steps: 1.1 The mobile robot collects the oil meter image through the telephoto lens at the first preset position; 1.2 Based on the deep learning target detection method, locate the oil meter in the image; 1.3 Based on the deep learning semantic segmentation method, the oil meter image is corrected by perspective transformation; 1.4 Based on the image enhancement processing method, the caliper position of the oil gauge is obtained; 1.5 Calculate the oil gauge reading based on the upper and lower limit position relationship between the caliper and the oil gauge; 1.6 Output oil meter reading; In step 1.3, the method based on deep learning semantic segmentation performs perspective transformation correction on the oil meter image, which can be divided into the following steps: 3.1 Expand the target detection area to obtain the semantic segmentation processing area; 3.2 Perform semantic segmentation on the target detection area to obtain segmentation masks of left and right straight lines and upper and lower arcs; 3.3 Perform point extraction and line fitting on the left and right straight line segmentation masks to obtain left and right straight line expressions; 3.4 Perform point extraction and binomial fitting on the upper and lower arc segmentation masks to obtain the quadratic expression of the upper and lower arcs; 3.5 Calculate the intersection points P1 and P2 of the left straight line and the upper and lower arcs; 3.5 Calculate the intersection points P3 and P4 of the right straight line and the upper and lower arcs; 3.6 Use the intersection points P1, P2, P3, and P4 to perform perspective transformation and obtain the image after perspective transformation; In step 1.4, a red channel image of the oil meter image corrected in step 1.3 is obtained by channel separation, and Gaussian filtering is performed. The contrast of the caliper in the image is enhanced by equalization. The caliper mask is obtained by image threshold and mask optimization, and the area with the largest area is selected as the position of the caliper.

2. The substation oil meter reading method based on deep learning according to claim 1 is characterized in that: In step 1.2, the target position of the oil gauge within the image range is obtained through a convolutional neural network, and the target is marked out of the image with a bounding box.

3. The substation oil meter reading method based on deep learning according to claim 2 is characterized in that: The convolutional neural network model is used to locate the position of the fuel gauge, specifically obtaining the detection frame of the target pair through classification recognition and regression calculation.

4. The substation oil meter reading method based on deep learning according to claim 1 is characterized in that: In step 3.3, after the left and right straight line points are extracted, the least squares method is used to fit the left and right straight line expressions.

5. The substation oil meter reading method based on deep learning according to claim 1 is characterized in that: In step 3.6, the perspective transformed image, i.e., the target rectangular area R ((rx1, ry1), (rx2, ry2)), is obtained through the intersection points P1, P2, P3, and P4, where (rx1, ry1) is the upper left corner coordinate of the target rectangular area, and (rx2, ry2) is the upper right corner coordinate of the target rectangular area, where: rx1=min(xp1,xp2,xp3,xp4), rx2=max(xp1,xp2,xp3,xp4), ry1=min(yp1,yp2,yp3,yp4), ry2=max(yp1,yp2,yp3,yp4), Among them, the intersection points of the left straight line and the upper and lower arcs are P1(xp1, yp1) and P2(xp2, yp2), the intersection points of the right straight line and the upper and lower arcs are P3(xp3, yp3) and P4(xp4, yp4), min() and max() are minimum and maximum value operations respectively.

6. The substation oil meter reading method based on deep learning according to claim 5 is characterized in that: In step 1.5, after obtaining the caliper position, the ratio of the caliper height position to the oil gauge height is calculated to obtain the meter reading.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the substation oil meter reading method based on deep learning are implemented as described in any one of claims 1-6.

8. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the substation oil meter reading method based on deep learning as described in any one of claims 1-6 are implemented.

Images