Telegraph pole perpendicularity detection method, evaluation method and device based on machine vision and storage medium

Abstract

The invention provides a rod perpendicularity detection method and device based on machine vision, an evaluation method and device and a storage medium. The detection method comprises the following steps: acquiring a first image and a second image of the same telegraph pole; acquiring a first segmentation image containing the telegraph pole in the first image and acquiring a second segmentation image containing the telegraph pole in the second image by adopting a neural network model subjected to recognition training on the telegraph pole; forming a first binary image based on the first segmented image, and forming a second binary image based on the second segmented image; obtaining a first contour of the telegraph pole and a second contour of the telegraph pole; and obtaining a first inclination angle of the minimum enclosing rectangle of the first contour and a second inclination angle of the minimum enclosing rectangle of the second contour, and calculating the verticality of the telegraph pole according to the first inclination angle and the second inclination angle. According to the detection method, the measurement efficiency can be effectively improved, and the accuracy of the detection result can be effectively improved.

Description

Technical Field

[0001] This application relates to the technical field of detecting the verticality of utility poles, and particularly relates to a method for detecting the verticality of poles, an evaluation method, a device, and a storage medium based on machine vision. Background Art

[0002] The power grid is an important part of the infrastructure of modern society and undertakes the tasks of power transmission and distribution. The reliable operation of the power grid depends on the coordinated work of multiple key components. Among them, utility poles are used to support electric wires and are an important part of the power transmission line. The verticality of utility poles will directly affect the stability and safety of power transmission. In some related technologies, detecting the verticality of utility poles depends on operators to detect the verticality of utility poles on site, and this detection method has the problem of low efficiency and requires a lot of manpower and material resources to complete the measurement task. In some related technologies, it is proposed to use machine vision technology to identify utility poles from the taken photos and analyze the verticality of utility poles in the photos, but the detection results are prone to large errors. Summary of the Invention

[0003] Based on this, it is necessary to provide a method for detecting the verticality of utility poles based on machine vision. This method for detecting the verticality of utility poles is based on machine vision technology and can improve the accuracy of detection results while achieving rapid detection of the verticality of utility poles.

[0004] According to some embodiments of the present application, a method for detecting the verticality of poles based on machine vision includes the following steps:

[0005] Obtain a first image and a second image of the same utility pole, and the shooting directions of the utility pole in the first image and the second image intersect;

[0006] Use a neural network model trained for identifying utility poles to identify the utility poles in the first image and the second image, obtain a first segmentation map containing the utility pole in the first image, and obtain a second segmentation map containing the utility pole in the second image;

[0007] Form a first binary map based on the first segmentation map, and form a second binary map based on the second segmentation map;

[0008] Find and filter the contours in the first binary map to obtain the first contour of the utility pole, and find and filter the contours in the second binary map to obtain the second contour of the utility pole;

[0009] Obtain a first tilt angle of the minimum circumscribed rectangle of the first contour and a second tilt angle of the minimum circumscribed rectangle of the second contour, and calculate the verticality of the utility pole according to the first tilt angle and the second tilt angle.

[0010] In some embodiments of the present application, the distance between the shooting point of the first image and the utility pole is equal to the distance between the shooting point of the second image and the utility pole, and the angle formed between the shooting direction of the utility pole in the first image and the shooting direction of the utility pole in the second image is 90° ± 5°, and the shooting directions of the first image and the second image are both parallel to the horizontal plane;

[0011] The first inclination angle is θ1, the second inclination angle is θ2, and the verticality of the utility pole is Θ. The verticality of the utility pole is calculated according to the following formula:

[0012]

[0013] In some embodiments of the present application, the steps of obtaining the first image and the second image of the same utility pole include:

[0014] Using a drone equipped with a rangefinder and a height sensor to take pictures of the utility pole, controlling the drone at a first position and taking the first image in a state parallel to the ground. The rangefinder is used to measure the distance between the drone and the utility pole, and the height sensor is used to measure the height of the drone relative to the ground;

[0015] Keeping the height of the drone unchanged and the distance between it and the utility pole unchanged, controlling the drone to move to a second position and taking the second image in a state parallel to the ground. The angle formed between the direction of the first position relative to the utility pole and the direction of the second position relative to the utility pole is 90° ± 5°.

[0016] In some embodiments of the present application, before using the neural network model to identify the utility pole in the first image and the second image, the following steps are further included:

[0017] Preprocessing the first image and the second image, and the preprocessing includes image enhancement processing, noise reduction processing, and size normalization processing.

[0018] In some embodiments of the present application, the step of forming the first binary image based on the first segmentation map includes: performing gray-scale processing on the first segmentation map, performing edge detection processing on the gray-scale processed first segmentation map, performing adaptive thresholding processing on the edge detection processed first segmentation map to form the first binary image; in the step of finding and screening the contours in the first binary image, screening is performed through the aspect ratio and area characteristics of the found contours;

[0019] The steps of forming the second binary image based on the second segmentation image include: performing grayscale processing on the second segmentation image, performing edge detection processing on the grayscale-processed second segmentation image, performing adaptive thresholding processing on the edge-detection-processed second segmentation image to form the second binary image; in the step of finding and filtering the contours in the second binary image, filtering is performed based on the aspect ratio and area characteristics of the found contours.

[0020] In some embodiments of the present application, the neural network model is a Faster RCNN model generated through the following steps:

[0021] Obtain an image dataset, where the image dataset includes a training set and a test set, and the images in the training set and the test set contain annotation information;

[0022] Use a pre-trained ResNet-101 residual neural network as the basic feature extraction network;

[0023] Retain the convolutional layer of the ResNet-101 model, remove the fully connected layer, and then add a region proposal network and a detection network to construct a Faster RCNN model;

[0024] Train the Faster RCNN model through the training set, and use the trained Faster RCNN model to predict the test set;

[0025] Compare the maximum localization error of the Faster RCNN model on the test set with a preset error threshold;

[0026] If the maximum localization error is less than the error threshold, use it as the neural network model; otherwise, select at least part of the images with a localization error greater than the error threshold as an incremental set, perform incremental training on the Faster RCNN model through the incremental set, and then repeat the steps of predicting the test set and comparing the maximum localization error with the preset error threshold until the maximum localization error is less than the error threshold.

[0027] In some embodiments of the present application, the steps of obtaining an image dataset include:

[0028] Use a drone to collect multiple training pictures of utility poles, and perform noise reduction processing, size normalization processing, and image enhancement processing on the multiple training pictures. When performing size normalization processing, keep the aspect ratio of the training pictures unchanged;

[0029] Divide the multiple training images into a training set and a test set, and use an annotation tool to manually annotate the training images in the training set and the test set to form corresponding annotation information.

[0030] According to some embodiments of the present application, a method for evaluating the construction quality of a pole erecting machine includes the following steps:

[0031] Use the method for detecting the pole verticality based on machine vision as described in any of the above embodiments to detect the pole verticality of multiple utility poles;

[0032] Evaluate the construction quality of the pole erecting machine according to the detected pole verticality of the utility poles.

[0033] According to some embodiments of the present application, a device for detecting the pole verticality based on machine vision, the device includes:

[0034] An image acquisition module, the image acquisition module is used to acquire a first image and a second image of the same utility pole, and the shooting directions of the utility pole in the first image and the second image intersect;

[0035] An image segmentation module, the image segmentation module is used to use a neural network model trained for identifying utility poles to identify the utility poles in the first image and the second image, and obtain a first segmentation map containing the utility pole in the first image and a second segmentation map containing the utility pole in the second image;

[0036] A contour screening module, the contour screening module is used to form a first binary image based on the first segmentation map, and search for and screen contours in the first binary image to obtain a first contour of the utility pole; the contour screening module is used to form a second binary image based on the second segmentation map, and search for and screen contours in the second binary image to obtain a second contour of the utility pole;

[0037] A verticality calculation module, the verticality calculation module is used to obtain a first inclination angle of the minimum circumscribed rectangle of the first contour and a second inclination angle of the minimum circumscribed rectangle of the second contour, and calculate the verticality of the utility pole according to the first inclination angle and the second inclination angle.

[0038] According to some embodiments of the present application, a storage medium stores a computer program, and when the computer program is executed by a processor, the processor executes the method for detecting the pole verticality based on machine vision as described in any of the above embodiments.

[0039] In the method for detecting the verticality of a utility pole based on machine vision in at least one of the above embodiments, a trained neural network model is used to identify the utility pole in the first image and the second image. The first segmentation map and the second segmentation map, the first binary map and the second binary map, and the first contour and the second contour are sequentially obtained. Finally, the verticality of the utility pole is calculated through the first inclination angle of the minimum circumscribed rectangle of the first contour and the second inclination angle of the minimum circumscribed rectangle of the second contour. This detection method can use machine vision for object detection and image processing technology to identify and process the captured pictures, so as to calculate the verticality of the utility pole, which can effectively improve the measurement efficiency and achieve rapid detection of the verticality of the utility pole. At the same time, this detection method uses two images with intersecting shooting directions as a reference, and combines the inclination angles of the utility pole in different directions to calculate the verticality of the utility pole, improving the problem of errors in the detection results caused by the influence of the shooting direction. Therefore, it can also effectively improve the accuracy of the detection results. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a schematic diagram of the steps of a method for detecting the verticality of a pole based on machine vision in an embodiment;

[0041] Figure 2 It is a training flow chart of a Faster RCNN model in an embodiment;

[0042] Figure 3 It is a calculation model diagram of the verticality of a utility pole in this application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0043] To facilitate the understanding of this application, the following will describe this application more comprehensively with reference to the relevant drawings. The preferred embodiments of this application are shown in the drawings. However, this application can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of this application more thorough and comprehensive.

[0044] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which this application belongs. The terms used in the specification of this application herein are only for the purpose of describing specific embodiments and are not intended to limit this application. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.

[0045] The purpose of the terms used herein is only to describe specific embodiments and is not a limitation of the present application. As used herein, the singular forms "a", "an" and "the" are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the terms "comprising" and / or "including", when used in this specification, determine the presence of features, integers, steps, operations, elements and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups. As used herein, the term "and / or" includes any and all combinations of the related listed items.

[0046] In some related technologies, it is proposed to use machine vision technology to identify utility poles from the captured photos and analyze their verticality. During the research process of the present application, it is found that the detection results of this analysis method are prone to large errors. This is mainly because the photo of the utility pole is the projection of the utility pole in three-dimensional space on a two-dimensional photo, and the verticality obtained from this photo is actually the verticality of the projection. The verticality of the projection is affected by the shooting direction and may not accurately reflect the true verticality of the utility pole in three-dimensional space. Therefore, the detection results are prone to errors.

[0047] The present application provides a method for detecting the verticality of a pole based on machine vision, which includes the following steps: obtaining a first image and a second image of the same utility pole, and the shooting directions of the utility pole in the first image and the second image intersect. Using a neural network model trained for identifying utility poles to identify the utility poles in the first image and the second image, obtaining a first segmentation map containing the utility pole in the first image, and obtaining a second segmentation map containing the utility pole in the second image. Forming a first binary map based on the first segmentation map, and forming a second binary map based on the second segmentation map. Searching for and screening the contours in the first binary map to obtain the first contour of the utility pole, and searching for and screening the contours in the second binary map to obtain the second contour of the utility pole. Obtaining the first inclination angle of the minimum circumscribed rectangle of the first contour and the second inclination angle of the minimum circumscribed rectangle of the second contour, and calculating the verticality of the utility pole according to the first inclination angle and the second inclination angle.

[0048] In the above-mentioned method for detecting the verticality of a utility pole based on machine vision in at least one of the embodiments, a trained neural network model is used to identify the utility pole in the first image and the second image. The first segmentation map and the second segmentation map, the first binary map and the second binary map, and the first contour and the second contour are sequentially obtained. Finally, the verticality of the utility pole is calculated through the first tilt angle of the minimum circumscribed rectangle of the first contour and the second tilt angle of the minimum circumscribed rectangle of the second contour. This detection method can use machine vision for object detection and image processing technology to identify and process the captured pictures to calculate the verticality of the utility pole, which can effectively improve the measurement efficiency and achieve rapid detection of the verticality of the utility pole. At the same time, this detection method uses two images with intersecting shooting directions as a reference, and combines the tilt angles of the utility pole in different directions to calculate the verticality of the utility pole, improving the problem of errors in the detection results caused by the influence of the shooting direction. Therefore, it can also effectively improve the accuracy of the detection results.

[0049] Figure 1 It is a schematic diagram of the steps of a method for detecting the verticality of a pole based on machine vision in an embodiment. Refer to Figure 1 As shown, an embodiment of the present application is a method for detecting the verticality of a pole based on machine vision. The method for detecting the verticality of the pole includes the following steps S1 to S5.

[0050] Step S1: Obtain a first image and a second image that capture the same utility pole.

[0051] Among them, the first image and the second image can be pre-captured images or on-site captured images. The shooting direction refers to the direction when the camera shoots the utility pole, and can also be understood as the direction where the connection line between the lighting element of the camera and the utility pole is located.

[0052] In some embodiments of the present application, the distance between the shooting position of the first image and the utility pole is equal to the distance between the shooting position of the second image and the utility pole. The angle formed between the shooting direction of the utility pole in the first image and the shooting direction of the utility pole in the second image is 90° ± 5°, and the shooting directions of the first image and the second image are both parallel to the horizontal plane. At this time, the utility poles in the first image and the second image only differ in the shooting direction, ensuring that the utility poles in the first image and the second image have almost the same or the same magnification ratio, standardizing and simplifying the subsequent calculation of the verticality of the utility pole.

[0053] In some embodiments of the present application, the method for photographing the first image and the second image includes the following steps: using a drone equipped with a rangefinder and an altimeter to photograph a utility pole, controlling the drone to be at a first position and photographing the first image in a state parallel to the ground; while keeping the height of the drone unchanged and the distance between the drone and the utility pole unchanged, controlling the drone to move to a second position and photographing the second image in a state parallel to the ground. The angle formed between the direction of the first position relative to the utility pole and the direction of the second position relative to the utility pole is 90° ± 5°.

[0054] It can be understood that the rangefinder is used to measure the distance between the drone and the utility pole, and the altimeter is used to measure the height of the drone relative to the ground. In some embodiments of the present application, the rangefinder uses a laser rangefinder. The altimeter uses an ultrasonic sensor.

[0055] In this embodiment, using a drone equipped with a rangefinder and an altimeter to photograph the same utility pole for the first image and the second image respectively can relatively accurately ensure that the height of the drone and the distance between the drone and the utility pole remain unchanged when taking the two images, avoiding the shooting errors caused by other accidental factors. In addition, using a drone can directly move in the air and can continuously photograph multiple utility poles, and the image acquisition efficiency is relatively high.

[0056] In some embodiments of the present application, the angle formed between the shooting direction of the first image and the shooting direction of the second image is 90° ± 5°. Among them, "±5°C" means that the angle can deviate by 5° relative to 90°. For example, the angle formed between the shooting direction of the first image and the shooting direction of the second image is 85°, 86°, 87°, 88°, 89°, 90°, 91°, 92°, 93°, 94°, 95°. Controlling the angle formed between the direction of the first position relative to the utility pole and the direction of the second position relative to the utility pole to be 90° ± 5° can ensure that the final detection result has relatively high accuracy.

[0057] In some embodiments of the present application, after step S1 and before step S2, the following steps are further included: preprocessing the first image and the second image. Among them, the preprocessing includes image enhancement processing, noise reduction processing, and size normalization processing. The main function of preprocessing the image is to reduce the interference of the background in the image on the subsequent recognition process, improve the accuracy of image segmentation, and further improve the accuracy of this detection method.

[0058] Noise reduction processing is used to reduce or eliminate noise in digital images. In some embodiments of the present application, when performing noise reduction processing, Gaussian filtering noise reduction can be used. It can be understood that Gaussian filtering noise reduction performs weighted averaging on the pixel values of the entire image. For each pixel value, it is obtained by weighted averaging its own value and other pixel values in the neighborhood.

[0059] Normalization processing is a process of adjusting images or data of different sizes to a unified size for subsequent processing and analysis. In some embodiments of the present application, when performing size normalization processing, the aspect ratio of the first image and the second image is kept unchanged.

[0060] It can be understood that noise has relatively little impact on the human eye's ability to distinguish images, but has a greater impact on the training and recognition processes of neural network models. When an image is stretched or shortened in a certain direction, the inclination angle of the utility poles in the image will also change significantly. There are usually relatively complex and different backgrounds in the pictures containing utility poles, and these backgrounds will also affect the accuracy of the neural network model in recognizing utility poles in the image. Before using the neural network model for recognition, the first image and the second image are first subjected to noise reduction processing, size normalization processing, and image enhancement processing. Its function is to ensure the success rate and accuracy of the neural network model in recognizing utility poles in the image, and reduce or eliminate errors.

[0061] Step S2: Use the neural network model trained for recognizing utility poles to recognize the utility poles in the first image and the second image, obtain the first segmentation map containing the utility poles in the first image, and obtain the second segmentation map containing the utility poles in the second image.

[0062] In some embodiments of the present application, in step S2, the neural network model is the Faster RCNN model generated through training. It can be understood that the Faster RCNN model is a deep learning model for object detection, which has efficient object detection capabilities and training speed. The Faster RCNN model is constructed based on the feature extraction network (Backbone Network).

[0063] In some embodiments of the present application, the pre-trained ResNet-101 residual neural network is used as the basic feature extraction network of the Faster RCNN model. The convolutional layer of the ResNet-101 model is retained, and the fully connected layer is removed. A region proposal network (Region Proposal Network) and a detection network are added on the basis of the modified ResNet-101 model to construct the Faster RCNN model for training.

[0064] Figure 2 is the training flowchart of the Faster RCNN model of an embodiment. Refer to Figure 2 As shown, in some embodiments of the present application, in step S2, the Faster RCNN model is trained through the following steps S2.1 to S2.6.

[0065] Step S2.1, obtain an image dataset. The image dataset includes a training set and a test set, and the images in the training set and the test set contain annotation information. Among them, the training set is used to be provided to an initial model for preliminary training so that the model can generate the ability to distinguish utility poles in images. The test set is used to be provided to the model trained with the training set for further training to improve the accuracy of the model in distinguishing utility poles.

[0066] In some embodiments of the present application, the ratio of the number of images in the training set to the number of images in the test set is (6 - 8):(2 - 4). For example, the ratio of the number of images in the training set to the number of images in the test set is 7:3.

[0067] In some embodiments of the present application, when obtaining the image dataset, the Image Labeler annotation tool in MATLAB is used to manually annotate the images in the training set and the test set to form corresponding annotation information.

[0068] In some embodiments of the present application, the step of obtaining the image dataset includes: using a drone to collect multiple training pictures of utility poles, and performing noise reduction processing, size normalization processing, and image enhancement processing on the multiple training pictures. The multiple training pictures are divided into a training set and a test set, and the training pictures in the training set and the test set are manually annotated using an annotation tool to form corresponding annotation information. It can be understood that the main functions of performing noise reduction processing, size normalization processing, and image enhancement processing on the training pictures are to highlight the utility poles from the complex environmental background, so that the neural network model trains on the processed images, and its function is to improve the recognition ability of the trained neural network model for images.

[0069] In some embodiments of the present application, Gaussian filtering noise reduction is used for noise reduction processing.

[0070] In some embodiments of the present application, when performing size normalization processing, the aspect ratio of the training pictures is kept unchanged.

[0071] In some embodiments of the present application, when collecting training images, the drone used for shooting is equipped with a rangefinder and an altitude sensor. Among multiple training images, the heights of the shooting positions can be the same or different, and the distances between the shooting positions and the utility poles can be the same or different. When the height and distance are the same, the trained model has a higher recognition rate for the utility poles photographed at that height and distance. When the height and distance are different, the trained model can distinguish images taken at various heights and distances, and has a wider applicability.

[0072] In some embodiments of the present application, the rangefinder uses a laser rangefinder. The altitude sensor uses an ultrasonic sensor.

[0073] Step S2.2: Use the pre-trained ResNet-101 residual neural network as the basic feature extraction network.

[0074] Step S2.3: Retain the convolutional layer of the ResNet-101 model, remove the fully connected layer, and then add a region proposal network and a detection network to construct a Faster RCNN model. Among them, ResNet-101 is a deep convolutional network and belongs to a type of residual neural network. In the research process of the present application, it is found that the Faster RCNN model constructed in the above manner based on the ResNet-101 residual neural network has extremely high recognition accuracy for utility poles in a complex environmental background, and can thus accurately segment the area where the utility poles are located from the image.

[0075] Step S2.4: Initially train the Faster RCNN model with a training set, use the initially trained Faster RCNN model to predict a test set, and then compare the maximum localization error of the Faster RCNN model on the test set with a preset error threshold; if the maximum localization error is less than the error threshold, use it as the neural network model.

[0076] It can be understood that the maximum localization error refers to the maximum localization error generated when the initially trained Faster RCNN model recognizes all images in the test set. The error threshold can be an artificially defined acceptable error value. When the maximum localization error of all images in the test set is less than the error threshold, it can be considered that the neural network model meets the requirements and the training is ended.

[0077] Step S2.5: If the maximum localization error is greater than the error threshold, select at least part of the images with a localization error greater than the error threshold as an incremental set, and incrementally train the Faster RCNN model with the incremental set.

[0078] Step S2.6: Use the incrementally trained Faster RCNN model to predict the test set, and then compare the maximum localization error of the Faster RCNN model on the test set with a preset error threshold. If the maximum localization error is less than the error threshold, use it as the neural network model; otherwise, repeat Step S2.5 and Step S2.6 until the maximum localization error is less than the error threshold.

[0079] In some embodiments of the present application, when selecting pictures as the incremental set, select the pictures with the top 10% localization errors as the incremental set.

[0080] It can be understood that through the above Steps S2.1 to S2.6, a trained neural network model can be obtained. In some embodiments of the present application, this neural network model can identify utility poles in an image and output the bounding box coordinates. These bounding box coordinates are used to extract the segmentation map containing the utility poles from the image.

[0081] Step S3: Form a first binary image based on the first segmentation map and a second binary image based on the second segmentation map.

[0082] In some embodiments of the present application, in Step S3, the step of forming the first binary image based on the first segmentation map includes: performing grayscale processing on the first segmentation map. Performing edge detection processing on the grayscale-processed first segmentation map. Performing adaptive thresholding processing on the edge-detection-processed first segmentation map to form the first binary image.

[0083] In some embodiments of the present application, when performing grayscale processing on the first segmentation map, the grayscale value is determined using the following formula: Gray = 0.299R + 0.587G + 0.114B. Where Gray is the grayscale value, R is the red channel value of the image, G is the green channel value of the image, and B is the blue channel value of the image.

[0084] In some embodiments of the present application, when performing edge detection processing on the grayscale-processed first segmentation map, the Canny edge detection algorithm is used for edge detection processing.

[0085] In some embodiments of the present application, after performing adaptive thresholding processing on the edge-detection-processed first segmentation map, it further includes: using morphological closing operation to eliminate defects in the binary image.

[0086] In some embodiments of the present application, in Step S3, the step of forming the second binary image based on the second segmentation map includes: performing grayscale processing on the second segmentation map. Performing edge detection processing on the grayscale-processed second segmentation map. Performing adaptive thresholding processing on the edge-detection-processed second segmentation map to form the second binary image.

[0087] In some embodiments of the present application, when performing grayscale processing on the second segmentation map, the grayscale value is determined using the following formula: Gray = 0.299R + 0.587G + 0.114B. Where Gray is the grayscale value, R is the red channel value of the image, G is the green channel value of the image, and B is the blue channel value of the image.

[0088] In some embodiments of the present application, when performing edge detection processing on the grayscale-processed second segmentation map, the Canny edge detection algorithm is used for edge detection processing.

[0089] In some embodiments of the present application, after performing adaptive thresholding processing on the edge-detected second segmentation map, it further includes: using morphological closing operation to eliminate defects in the binary map.

[0090] Step S4, find and filter the contours in the first binary map to obtain the first contour of the utility pole, and find and filter the contours in the second binary map to obtain the second contour of the utility pole.

[0091] In some embodiments of the present application, in step S4, in the step of finding and filtering the contours in the first binary map, filtering is performed based on the aspect ratio and area characteristics of the found contours.

[0092] In some embodiments of the present application, in step S4, in the step of finding and filtering the contours in the second binary map, filtering is performed based on the aspect ratio and area characteristics of the found contours.

[0093] Step S5, obtain the first tilt angle of the minimum circumscribed rectangle of the first contour and the second tilt angle of the minimum circumscribed rectangle of the second contour, and calculate the verticality of the utility pole according to the first tilt angle and the second tilt angle.

[0094] In some embodiments of the present application, in step S5, the first tilt angle is θ1, the second tilt angle is θ2, and the verticality of the utility pole is Θ. The verticality of the utility pole is calculated according to the following formula:

[0095] Specifically, Figure 3 is the calculation model diagram of the verticality of the utility pole in the present application. Refer to Figure 3As shown in the figure, rod AB represents the actual utility pole, and the verticality of the actual utility pole is Θ. Rod A1B1 represents the projection of rod AB when the first image is taken. Rod A1B1 has a first inclination angle, which is θ1. Rod A2B2 represents the projection of rod AB when the second image is taken. Rod A2B2 has a first inclination angle, which is θ2. It can be found that both the first inclination angle and the second inclination angle have a certain gap with the actual verticality Θ. That is to say, the inclination angle of the utility pole contour obtained from a single utility pole image cannot accurately reflect the actual verticality of the utility pole. After noticing this problem, the present application proposes a method for detecting the verticality of a pole based on machine vision in the present application.

[0096] Referring to Figure 3 As shown in the figure, in some embodiments, the direction of the shooting point of the first image relative to the utility pole and the direction of the shooting point of the second image relative to the utility pole are perpendicular to each other. At this time, the planes recorded by the first image and the second image are perpendicular to each other. Therefore, according to the Pythagorean theorem, there is the following geometric relationship:

[0097] tan 2 Θ = tan 2 (θ1) + tan 2 (θ2).

[0098] After further adjustment, the following formula can be obtained:

[0099]

[0100] In summary, the verticality of the utility pole can be calculated using the above formula. Since this verticality combines the images in two shooting directions, the resulting error is smaller and it is closer to the actual verticality of the utility pole. In addition, in this embodiment, the calculation method of the verticality is related to the included angle between the shooting directions of the first image and the second image. When the shooting directions of the first image and the second image are not in a vertical state, the verticality of the utility pole can be calculated according to the cosine theorem.

[0101] It can be understood that through the above steps S1 to S5, the method for detecting the verticality of a pole based on machine vision in the present application can be completed.

[0102] Furthermore, the present application also provides an evaluation method for the construction quality of a pole erecting machine. The evaluation method includes the following steps: detecting the pole verticality of multiple utility poles by using the method for detecting the verticality of a pole based on machine vision in the above embodiment; evaluating the construction quality of the pole erecting machine according to the detected pole verticality of the multiple utility poles.

[0103] It can be understood that the method for detecting the pole verticality based on machine vision in this application can efficiently and accurately detect the verticality of multiple utility poles. Therefore, by using this method for detecting the pole verticality to detect the verticality of multiple utility poles, the statistical results of the verticality of the multiple utility poles can reflect the construction quality of the pole erecting machine.

[0104] In some embodiments, the parameters for evaluating the construction quality of the pole erecting machine include but are not limited to: the maximum value of the verticality of multiple utility poles, the average value of the verticality of multiple utility poles, the variance or standard deviation of the verticality of multiple utility poles, etc.

[0105] Utility poles are usually installed using a pole erecting machine, and the construction quality of the pole erecting machine directly affects the installation quality of the utility poles. In order to improve the safety of the power grid and reduce maintenance costs, it is necessary to monitor the construction quality of the pole erecting machine. The verticality of the utility pole is an important factor for evaluating the construction quality of the pole erecting machine. If the method of on-site measurement of the verticality of the utility pole by an operator in the related art is adopted, there are problems of time-consuming and laborious, and it is difficult to timely feedback the construction quality of the pole erecting machine. If the method of analyzing the verticality of the utility pole by machine vision technology in the related art is adopted, there are also large errors, and it is difficult to accurately evaluate the construction quality of the pole erecting machine.

[0106] This application detects the pole verticality through machine vision technology, and obtains the verticality of the utility pole based on the first image and the second image during the detection, which can solve the problems of low efficiency and large errors at the same time. Therefore, this method for detecting the pole verticality can be used to timely and accurately evaluate the construction quality of the pole erecting machine, and further can realize the monitoring of the construction quality of the pole erecting machine.

[0107] This application also provides a device for detecting the pole verticality based on machine vision. The device includes: an image acquisition module, an image segmentation module, a contour screening module, and a verticality calculation module. Among them, the image acquisition module is used to acquire the first image and the second image of the same utility pole, and the shooting directions of the utility pole in the first image and the second image intersect. The image segmentation module is used to identify the utility pole in the first image and the second image by using a neural network model trained for identifying the utility pole, and obtain the first segmentation map containing the utility pole in the first image and the second segmentation map containing the utility pole in the second image. The contour screening module is used to form a first binary image based on the first segmentation map, and search for and screen the contours in the first binary image to obtain the first contour of the utility pole; the contour screening module is used to form a second binary image based on the second segmentation map, and search for and screen the contours in the second binary image to obtain the second contour of the utility pole. The verticality calculation module is used to obtain the first inclination angle of the minimum circumscribed rectangle of the first contour and the second inclination angle of the minimum circumscribed rectangle of the second contour, and calculate the verticality of the utility pole according to the first inclination angle and the second inclination angle.

[0108] It can be understood that when the rod verticality detection device based on machine vision provided in the above embodiments performs rod verticality detection, only the division of the above program modules is used for illustration. In practical applications, the above processing can be allocated to different program modules according to needs, that is, the internal structure of the device is divided into different program modules to complete all or part of the above-described processing. In addition, the rod verticality detection device based on machine vision provided in the above embodiments and the rod verticality detection method based on machine vision described in the prior embodiments belong to the same concept. For the specific implementation process, please refer to the method embodiments and will not be elaborated here.

[0109] The present application also provides a storage medium storing a computer program, which, when executed by a processor, causes the processor to execute the rod verticality detection method based on machine vision according to any one of the above embodiments.

[0110] In some embodiments of the present application, the storage medium is a computer-readable storage medium. The storage medium may be a ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface memory, optical disc, or CD-ROM, etc.

[0111] The above detailed description is a specific description of the feasible embodiments of the present application. These embodiments are not intended to limit the patent scope of the present application. Any equivalent implementation or modification without departing from the present application shall be included in the patent scope of this case.

Claims

1. A method for detecting verticality of a pole based on machine vision, characterized in that: The steps include: Acquire a first image and a second image that capture the same utility pole, where shooting directions of the utility pole in the first image and the second image intersect; Using a neural network model trained for identifying utility poles, identifying the utility poles in the first image and the second image, obtaining a first segmentation map containing the utility poles in the first image, and obtaining a second segmentation map containing the utility poles in the second image; forming a first binary image based on the first segmentation image, and forming a second binary image based on the second segmentation image; Searching for and filtering the contour in the first binary image to obtain a first contour of the utility pole, and searching for and filtering the contour in the second binary image to obtain a second contour of the utility pole; A first tilt angle of a minimum circumscribed rectangle of the first contour and a second tilt angle of a minimum circumscribed rectangle of the second contour are obtained, and the verticality of the utility pole is calculated according to the first tilt angle and the second tilt angle.

2. The method for detecting the perpendicularity of a pole based on machine vision according to claim 1, wherein, The distance between the shooting point of the first image and the utility pole is equal to the distance between the shooting point of the second image and the utility pole, the angle formed by the shooting direction of the utility pole in the first image and the shooting direction of the utility pole in the second image is 90°±5°, and the shooting direction of the first image and the shooting direction of the second image are both parallel to the horizontal plane; The first tilt angle is θ1, the second tilt angle is θ2, and the verticality of the utility pole is Θ. The verticality of the utility pole is calculated according to the following formula:

3. The method for detecting the perpendicularity of a pole based on machine vision according to claim 2, wherein, The steps of acquiring a first image and a second image of the same utility pole include: Using a drone equipped with a rangefinder and an altimeter to photograph the utility pole, the drone is controlled to capture the first image at a first location and in a state parallel to the ground, the rangefinder is used to measure the distance of the drone relative to the utility pole, and the altimeter is used to measure the height of the drone relative to the ground; Maintaining a constant height of the UAV and a constant distance between the UAV and the utility pole, the UAV is controlled to move to a second position and capture the second image while being parallel to the ground, wherein an angle between a direction of the first position relative to the utility pole and a direction of the second position relative to the utility pole is 90°±5°.

4. The method for detecting verticality of a rod based on machine vision according to any one of claims 1 to 3, characterized in that: Before using the neural network model to identify the utility poles in the first image and the second image, the method further includes the following steps: Preprocessing is performed on the first image and the second image, where the preprocessing includes image enhancement processing, noise reduction processing, and size normalization processing.

5. The rod verticality detection method based on machine vision according to any one of claims 1 to 3, characterized in that: The steps of forming the first binary image based on the first segmentation image include: performing grayscale processing on the first segmentation image, performing edge detection processing on the grayscale-processed first segmentation image, performing adaptive thresholding processing on the edge-detection-processed first segmentation image to form the first binary image; in the step of finding and screening contours in the first binary image, screening is performed based on the aspect ratio and area characteristics of the found contours; The steps of forming the second binary image based on the second segmentation image include: performing grayscale processing on the second segmentation image, performing edge detection processing on the grayscale-processed second segmentation image, performing adaptive thresholding processing on the edge-detection-processed second segmentation image to form the second binary image; in the step of finding and screening contours in the second binary image, screening is performed based on the aspect ratio and area characteristics of the found contours.

6. The method for detecting the perpendicularity of a rod based on machine vision according to any one of claims 1 to 3, characterized in that, The neural network model is a Faster RCNN model generated through the following steps: Obtain an image dataset, where the image dataset includes a training set and a test set, and the images in the training set and the test set contain annotation information; Use a pre-trained ResNet-101 residual neural network as the basic feature extraction network; Retain the convolutional layer of the ResNet-101 model, remove the fully connected layer, and then add a region proposal network and a detection network to construct a Faster RCNN model; Train the Faster RCNN model through the training set, and use the trained Faster RCNN model to predict the test set; Compare the maximum localization error of the Faster RCNN model on the test set with a preset error threshold; If the maximum localization error is less than the error threshold, use it as the neural network model; Otherwise, select at least part of the images with a localization error greater than the error threshold as an incremental set, perform incremental training on the Faster RCNN model through the incremental set, and then repeat the steps of predicting the test set and comparing the maximum localization error with the preset error threshold until the maximum localization error is less than the error threshold.

7. The method for detecting the perpendicularity of a rod based on machine vision according to claim 6, characterized in that, The steps of obtaining the image dataset include: Use a drone to collect multiple training images of utility poles, and perform noise reduction processing, size normalization processing, and image enhancement processing on the multiple training images. When performing size normalization processing, keep the aspect ratio of the training images unchanged; Divide the multiple training images into a training set and a test set, and use an annotation tool to perform manual annotation on the training images in the training set and the test set to form corresponding annotation information.

8. An evaluation method for the construction quality of a pole erecting machine, characterized in that, Include the following steps: Use the machine vision-based pole verticality detection method described in any one of claims 1 to 7 to detect the pole verticality of multiple utility poles; Evaluate the construction quality of the pole erecting machine according to the detected pole verticality of the multiple utility poles.

9. A rod verticality detection device based on machine vision, characterized in that: The device includes: An image acquisition module, which is used to acquire a first image and a second image of the same utility pole, and the shooting directions of the utility pole in the first image and the second image intersect; An image segmentation module, which is used to identify the utility pole in the first image and the second image by using a neural network model trained for utility pole recognition, and obtain a first segmentation map containing the utility pole in the first image and a second segmentation map containing the utility pole in the second image; A contour screening module, which is used to form a first binary image based on the first segmentation map, find and screen contours in the first binary image to obtain a first contour of the utility pole; the contour screening module is used to form a second binary image based on the second segmentation map, and find and screen contours in the second binary image to obtain a second contour of the utility pole; A verticality calculation module, which is used to obtain a first inclination angle of the minimum circumscribed rectangle of the first contour and a second inclination angle of the minimum circumscribed rectangle of the second contour, and calculate the verticality of the utility pole according to the first inclination angle and the second inclination angle.

10. A storage medium, characterized in that, A computer program is stored in the storage medium, and when the computer program is executed by a processor, the processor is caused to execute the method for detecting the verticality of a pole based on machine vision according to any one of claims 1 to 7.

Images