A method for inspecting conductors of ultra-high voltage lines based on unmanned aerial vehicle simulation inspection

Abstract

The invention relates to the technical field of power transmission line operation and maintenance, and discloses an ultra-ultra-high voltage line conductor inspection method based on unmanned aerial vehicle line inspection, comprising: taking a ground conductor between a starting pole tower and a destination pole tower as an inspection unit, distinguishing each power line in each ground conductor in the current frame image based on each frame image in a video stream collected by the unmanned aerial vehicle in real time, merging detection frames corresponding to two adjacent power lines into a candidate frame in pairs according to the positions of the multiple power lines from top to bottom in the current frame image, and taking the candidate frame in the middle position as a target frame; at the moment of each frame image in the video stream collected by the unmanned aerial vehicle in real time, taking the current frame image target frame as a reference, calculating in real time the height difference between the center of the target frame and the center of the current frame image frame, adjusting the working angle of the unmanned aerial vehicle, keeping the center of the current frame image target frame highly consistent with the center of the current frame image frame, and inspecting the ground conductor corresponding to the current inspection unit.

Description

Technical Field

[0001] The present invention relates to the technical field of power transmission line operation and maintenance, and in particular to an ultra-high voltage line conductor inspection method based on unmanned aerial vehicle (UAV) line inspection. Background Art

[0002] Traditional overhead transmission line ground wires are exposed to complex and changeable natural environments for a long time, such as continuous wind, severe rain and snow, electrical stress, material fatigue and other factors, which can easily lead to wire breakage, rust, wear and corrosion. If these problems are not discovered and effectively handled in time, they are likely to cause large-scale power outages and cause immeasurable economic losses.

[0003] The traditional manual inspection mode can no longer meet the needs of modern power grid management. With the continuous advancement and deepening of drone inspection replacing manual inspection, it has brought innovation to line inspection. Drone inspection has shown significant advantages in the inspection of fixed-point components; but in the inspection of long-distance, continuous linear components such as ground wires, the main inspection method is manual control of drones. The collection process is chaotic, easy to miss or take the wrong picture, and the inspection efficiency and quality depend on the pilot's skills, which makes it impossible to conduct stable and efficient ground wire inspections. At the same time, when facing ground wires of different voltages, due to the different numbers and types of wires, drone inspections are difficult to track, and it is difficult to ensure continuous tracking of the same ground wire.

[0004] In addition, the positioning accuracy of drones is limited. The existing methods use point cloud data to plan cruise paths and assist manually controlled drone inspections, but it is difficult to plan the best inspection path, and the point cloud route cannot automatically adapt to changes in sag, resulting in difficulty for drone inspections to fly along the sag. The lack of high-precision autonomous positioning and visual navigation tracking makes it impossible to accurately identify and track sag, which in turn makes it impossible to conduct a comprehensive inspection of the ground wire and causes the sag portion to be missed.

[0005] In summary, the existing method of using drones to inspect ground wires has poor tracking accuracy for ground wires and cannot accurately identify the sag of ground wires, which may lead to omissions in inspections and result in weak inspection and investigation of ground wires; and there is a lack of effective paths for inspecting poles and towers. Summary of the invention

[0006] To this end, the technical problem to be solved by the present invention is to overcome the problem in the prior art that the ground wire cannot be accurately tracked during drone inspections, resulting in poor inspection accuracy for the ground wire.

[0007] In order to solve the above technical problems, the present invention provides an ultra-high voltage line conductor inspection method based on unmanned aerial vehicle line inspection, comprising:

[0008] Take any two adjacent towers in the area to be inspected as the starting tower and the destination tower, obtain the coordinates of the starting tower and the destination tower, and use them as the inspection starting point and inspection destination point of the drone respectively;

[0009] Based on a three-dimensional point cloud image including a starting tower, a target tower, and multiple ground wires between the starting tower and the target tower, the multiple ground wires are obtained, one of the ground wires is used as an inspection unit, and each ground wire includes multiple split power lines;

[0010] Based on each frame of the video stream collected by the drone in real time, the target edge of each power line in each ground wire in the current frame image is identified to distinguish each power line;

[0011] According to the positions of the multiple power lines from top to bottom in the current frame image, the detection frames corresponding to two adjacent power lines are merged into one candidate frame, and multiple candidate frames arranged from top to bottom in the current frame image are obtained; the candidate frame in the middle position is used as the target frame of the current frame image;

[0012] At each frame of the video stream collected by the drone in real time, the target frame of the current frame is used as a reference to calculate the height difference between the center of the target frame of the current frame and the center of the frame of the current frame in real time. The working angle of the drone is adjusted to keep the center of the target frame of the current frame consistent with the center of the frame of the current frame, and inspect a ground wire corresponding to the current inspection unit.

[0013] Preferably, the method of identifying the target edge of each power line in each ground wire in the current frame image and distinguishing each power line based on each frame image in the video stream collected in real time by the drone includes:

[0014] For each frame of the video stream collected by the drone in real time, the Canny operator is used to extract the initial edges of all power lines in the current frame.

[0015] Based on the initial edge of each power line, multiple straight line segments are extracted using Hough transform;

[0016] Calculate the slope of each straight line segment and group the straight line segments with equal slopes into a group;

[0017] In each combination, the straight line segments belonging to the same power line in each combination are merged based on the longitudinal distance values ​​and the Euclidean distance between different straight line segments;

[0018] The merged target straight line segment in each combination is obtained as the target edge of the power line represented by each combination, and each power line in the current frame image is distinguished.

[0019] Preferably, for each frame of the video stream collected by the drone in real time, using the Canny operator to extract the initial edges of all power lines in the current frame image includes:

[0020] Use Gaussian function to denoise the current frame image to obtain a smooth image;

[0021] The gradient magnitude and direction at each pixel point on the smoothed image are calculated using the first-order differential, and the gradient magnitude is non-maximum suppressed to obtain the gradient magnitude at each pixel point;

[0022] Compare the gradient magnitude at each pixel with the preset high threshold and the preset low threshold:

[0023] If the gradient amplitude at the pixel point is greater than a preset high threshold, the pixel point is determined to be an edge point;

[0024] If the gradient amplitude at the pixel point is less than the preset low threshold, it is determined that the pixel point is not an edge point;

[0025] If the gradient amplitude at the pixel point is not greater than the preset high threshold and not less than the preset low threshold, then determine whether there is an edge point among the adjacent pixels of the pixel point:

[0026] If there is an edge point, then the pixel point is determined to be an edge point;

[0027] If there is no edge point, it is determined that the pixel point is not an edge point;

[0028] All edge points are connected in sequence to obtain the initial edges of all power lines in the current frame image.

[0029] Preferably, in each combination, based on the longitudinal distance values ​​and the Euclidean distances between different straight line segments, merging the straight line segments belonging to the same power line in each combination comprises:

[0030] In each combination, the longitudinal distance between any two straight line segments is calculated respectively;

[0031] Get the two straight line segments with the smallest longitudinal distance value, and compare the longitudinal distance values ​​of these two straight line segments with the preset longitudinal distance:

[0032] If the longitudinal distance between the two straight line segments is not less than the preset longitudinal distance, it is determined that the two straight line segments do not belong to the same power line;

[0033] If the longitudinal distance between the two straight line segments is less than the preset longitudinal distance, the Euclidean distances between the four endpoints of the two straight line segments are calculated, and the smallest Euclidean distance is compared with the preset transverse distance:

[0034] If the minimum Euclidean distance is not less than the preset horizontal distance, it is determined that the two straight line segments do not belong to the same electric line;

[0035] If the minimum Euclidean distance is less than the preset horizontal distance, it is determined that the two straight line segments belong to the same electric line; and the two end points with the minimum Euclidean distance are used as connection points to connect the two straight line segments to obtain the candidate straight line segments of the combination;

[0036] The longitudinal distance value and the Euclidean distance between the candidate straight line segment of the combination and any straight line segment in the remaining straight line segments in the combination are calculated, the straight line segments belonging to the same power line as the candidate straight line segment are connected, and the candidate straight line segments are updated until there is no straight line segment in the combination whose longitudinal distance value to the candidate straight line segment is less than the preset longitudinal distance, and the current candidate straight line segment is output as the target straight line segment of the combination.

[0037] Preferably, calculating the longitudinal distance value between any two straight line segments includes:

[0038] Calculate the shortest distances from the two endpoints of the first straight line segment to the straight line where the second straight line segment is located, and record them as the first distance and the second distance respectively;

[0039] Calculate the shortest distances from the two endpoints of the second straight line segment to the straight line where the first straight line segment is located, and record them as the third distance and the fourth distance respectively;

[0040] An average of the first distance, the second distance, the third distance, and the fourth distance is calculated as a longitudinal distance value between the first straight line segment and the second straight line segment.

[0041] Preferably, taking the candidate frame in the middle position as the target frame of the current frame image includes:

[0042] If the total number of candidate boxes If it is singular, the The candidate box is the target box of the current frame image; Indicates rounding up;

[0043] If the total number of candidate boxes If it is an even number, then the The candidate box is the target box of the current frame image.

[0044] Preferably, after obtaining the target frame of the current frame image, the method further includes:

[0045] Calculate the intersection-and-union ratio of the target frame in the current frame image and the target frame in the previous frame image, and determine whether the intersection-and-union ratio is greater than zero:

[0046] If the intersection-and-union ratio is not greater than zero, the target frame in the previous frame image is used as the target frame in the current frame image;

[0047] If the IoU ratio is greater than zero, images of a preset number of frames before the current frame image are obtained, the IoU ratios between adjacent frame images are calculated and sorted, and the target frame in the image with the largest IoU ratio is used as the target frame in the current frame image.

[0048] Preferably, taking the target frame of the current frame image as a reference, calculating in real time the height difference between the center of the target frame of the current frame image and the center of the frame of the current frame image, and adjusting the working angle of the drone, including:

[0049] If the height difference between the center of the target frame of the current frame image and the center of the frame of the current frame image is not greater than the preset height difference, the working angle of the drone gimbal is adjusted so that the center of the target frame of the current frame image is highly consistent with the center of the frame of the current frame image collected by the drone;

[0050] If the height difference between the center of the target frame of the current frame image and the center of the frame of the current frame image is greater than the preset height difference, the gimbal of the drone is returned to the center, and the drone is moved up and down by adjusting the flight control so that the center of the target frame of the current frame image is highly consistent with the center of the frame of the current frame image collected by the drone.

[0051] Preferably, after completing the inspection of the ground wire, the method further includes calculating the maximum sag of the ground wire between the starting tower and the destination tower based on the flight path of the drone, which is expressed as:

[0052] ;

[0053] in, Indicates the maximum sag of the ground wire between the starting tower and the destination tower. Indicates the height from the connection point between the ground wire and the starting tower to the ground. Indicates the height from the connection point between the ground wire and the destination tower to the ground. Indicates the actual flight altitude of the drone at the midpoint between the starting tower and the destination tower.

[0054] Preferably, after completing the inspection of the ground wire, it also includes: obtaining a current frame image collected during the drone inspection process, using a deep learning model to detect the current frame image, and identifying ground wire defects.

[0055] Preferably, the inspection of the tower includes:

[0056] The drone starts from the connection point on one end of the lowest phase cross arm on one side of the tower, moves along the direction perpendicular to the cross arm to the connection point on the other end of the lowest phase cross arm, and detects the power lines within the range of the two connection points on the lowest phase cross arm of the tower;

[0057] After the detection of the lowest phase cross arm on one side of the tower is completed, the drone is ordered to move upward in a bow-shaped cruising path, and detect the power lines within the range of two connection points on the remaining phase cross arms on one side of the tower until the detection of the highest phase cross arm is completed;

[0058] At the uppermost horizontal arm, along the direction parallel to the uppermost horizontal arm, from one side of the tower, across the tower, to the other side of the tower;

[0059] On the other side of the tower, the drone starts from the connection point on one end of the uppermost horizontal arm on the other side of the tower, and moves toward the connection point on the other end of the uppermost horizontal arm in a direction perpendicular to the horizontal arm, and detects the power lines within the range of the two connection points on the uppermost horizontal arm of the tower;

[0060] After the inspection of the top phase cross arm on the other side of the tower is completed, the drone is ordered to move downward in a bow-shaped cruising path to inspect the power lines within the range of two connection points on the remaining phase cross arms on the other side of the tower until the inspection of the bottom phase cross arm is completed, completing the inspection of the tower.

[0061] Preferably, it also includes:

[0062] According to the distance between the starting tower and the destination tower, the number of ground wires between the starting tower and the destination tower, the preset inspection speed of the drone, and the time required for inspecting the tower, the minimum flight distance of the drone for the drone to complete the inspection of the starting tower and the inspection of the ground wire between the starting tower and the destination tower is obtained, which is expressed as: ;

[0063] in, Represents the distance between the starting tower and the destination tower, expressed as , , and They represent the horizontal, vertical and vertical coordinates of the tower in the spatial rectangular coordinate system respectively. represents the coordinates of the starting tower, Indicates the coordinates of the target tower; Indicates the number of ground wires between the starting tower and the destination tower. Indicates the time required for tower inspection. Indicates the preset inspection speed of the drone; Indicates the minimum flight distance of the drone;

[0064] Based on the minimum flight distance of the drone and the preset inspection speed of the drone, obtain the minimum flight time of the drone , expressed as: .

[0065] The above technical solution of the present invention has the following beneficial effects compared with the prior art:

[0066] The ultra-ultra-high voltage line conductor inspection method based on unmanned aerial vehicle line inspection of the present invention takes a ground wire as an inspection unit, distinguishes each power line based on the target edge of multiple power lines in the ground wire, and merges the power lines into a candidate frame in pairs according to the position order from top to bottom, and then takes the candidate frame in the middle position as the target frame; using the identified target frame as a reference, the unmanned aerial vehicle flies up and down during the inspection process, so that the target frame of the current frame image is located in the middle of the frame of the current frame image, even if there is sag, it can ensure that the flying height of the unmanned aerial vehicle is always consistent with the height of the ground wire, thereby realizing accurate positioning and tracking of the ground wire, improving the inspection accuracy and efficiency, and making the inspection work more autonomous, convenient and safe.

[0067] Based on the extraction of edges by the Canny operator and the extraction of straight line segments by the Hough transform, the present invention proposes a line feature recovery method based on the grouping and merging of straight line segments. The identified straight line segments are divided into different groups according to the slope, and different groups represent different power lines. In each group, two straight line segments that meet the conditions are connected based on the longitudinal distance value and the Euclidean distance between the straight line segments, thereby obtaining the target edge of the power line represented by the combination. The present invention further splices the identified straight line segments based on the distance between the straight line segments, thereby improving the accuracy of power line edge recognition, thereby improving the accuracy of ground wire tracking during drone inspections.

[0068] The present invention divides the inspection into two parts: the pole tower and the ground wire. When inspecting the pole tower, each phase cross arm on both sides of the pole tower is inspected layer by layer in a bow-shaped patrol path, which can ensure that each layer of the cross arm and both sides of the pole tower are comprehensively inspected, and unnecessary round trips and repeated paths are reduced, thus saving inspection time and improving inspection efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] In order to make the content of the present invention more clearly understood, the present invention is further described in detail below according to specific embodiments of the present invention in conjunction with the accompanying drawings, wherein:

[0070] Figure 1 It is a flowchart of the steps of the ultra-high voltage line conductor inspection method based on the unmanned aerial vehicle line inspection provided by the present invention;

[0071] Figure 2 It is a flow chart of the steps of power line target edge identification;

[0072] Figure 3 It is a schematic diagram of the tower inspection route;

[0073] Figure 4 It is a schematic diagram of sag;

[0074] Figure 5It is a schematic diagram of the inspection system structure;

[0075] Figure 6 It is a functional architecture diagram of the inspection system;

[0076] Figure 7 is a flow chart of the complete power line edge restoration steps;

[0077] Figure 8 This is a schematic diagram of the calculation of the vertical spacing of a straight line segment;

[0078] Fig. 9 This is a schematic diagram of the overall structure of DeepLabV3;

[0079] Fig.10 This is a schematic diagram of the encoder-decoder structure of the DeepLabV3+ network;

[0080] Fig.11 It is a flow chart of ground wire defect model training and identification;

[0081] Fig.12 This is a schematic diagram comparing the recognition accuracy of different algorithms;

[0082] Fig.13 It is the effect diagram of defect identification;

[0083] Fig.14 This is a schematic diagram for identifying the ground wire. DETAILED DESCRIPTION

[0084] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.

[0085] Reference Figure 1 As shown, the flowchart of the method for inspecting the conductor of an ultra-high voltage line based on unmanned aerial vehicle line inspection provided by the present invention specifically includes the following steps:

[0086] S101: Taking any two adjacent poles in the area to be inspected as the starting pole and the destination pole, obtaining the coordinates of the starting pole and the destination pole, and using them as the inspection starting point and the inspection destination point for the drone to inspect respectively;

[0087] S102: based on a three-dimensional point cloud image including a starting tower, a destination tower, and multiple ground wires between the starting tower and the destination tower, the multiple ground wires are acquired, one of the ground wires is used as an inspection unit, and each ground wire includes multiple split power lines;

[0088] S103: Based on each frame of the video stream collected by the drone in real time, identify the target edge of each power line in each ground wire in the current frame of the image, and distinguish each power line;

[0089] S104: according to the positions of the plurality of power lines from top to bottom in the frame of the current frame image, the detection frames corresponding to two adjacent power lines are merged into one candidate frame in pairs, and a plurality of candidate frames arranged from top to bottom in the frame of the current frame image are obtained; and the candidate frame in the middle position is used as the target frame of the current frame image;

[0090] S105: At each frame of the video stream collected by the drone in real time, the target frame of the current frame is used as a reference, and the height difference between the center of the target frame of the current frame and the center of the frame of the current frame is calculated in real time, and the working angle of the drone is adjusted to keep the center of the target frame of the current frame consistent with the center of the frame of the current frame, and inspect a ground wire corresponding to the current inspection unit.

[0091] The ultra-ultra-high voltage line conductor inspection method based on unmanned aerial vehicle line inspection of the present invention takes a ground wire as an inspection unit, distinguishes each power line based on the target edge of multiple power lines in the ground wire, and merges the power lines into a candidate frame in pairs according to the position order from top to bottom, and then takes the candidate frame in the middle position as the target frame; using the identified target frame as a reference, the unmanned aerial vehicle flies up and down during the inspection process, so that the target frame of the current frame image is located in the middle of the frame of the current frame image, even if there is sag, it can ensure that the flying height of the unmanned aerial vehicle is always consistent with the height of the ground wire, thereby realizing accurate positioning and tracking of the ground wire, improving the inspection accuracy and efficiency, and making the inspection work more autonomous, convenient and safe.

[0092] In drone inspection operations, accurate identification of the location and path of power lines plays a decisive role in ensuring the accuracy of drone inspection routes. Therefore, whether the power lines can be correctly identified is extremely critical. In aerial images, the power lines are difficult to extract because the color of the power lines is similar to the background under a complex background. To this end, this embodiment first uses the Canny operator to perform image filtering based on the geometric characteristics of the power lines, then extracts the edges based on gradient calculation, and then uses the Hough transform to extract the straight line segments in the image. Finally, the complete power lines are restored using the combination and merging algorithm based on the extracted straight line segments. Figure 2 FIG. 1 is a flowchart of the steps of power line target edge recognition. Specifically, in step S103, each power line in each ground wire in the current frame image is recognized, including:

[0093] S103-1: For each frame of the video stream collected by the drone in real time, the Canny operator is used to extract the initial edges of all power lines in the current frame;

[0094] S103-2: Based on the initial edge of each power line, a plurality of straight line segments are extracted using Hough transform;

[0095] S103-3: Calculate the slope of each straight line segment, and divide the straight line segments with equal slopes into a group;

[0096] S103-4: In each combination, based on the longitudinal distance values ​​and the Euclidean distances between different straight line segments, the straight line segments belonging to the same power line in each combination are merged;

[0097] S103-5: Obtain the merged target straight line segment in each combination as the target edge of the power line represented by each combination, and distinguish each power line in the current frame image.

[0098] Specifically, in step S103-1, the initial edges of all power lines in the current frame image are extracted using the Canny operator, including:

[0099] Use Gaussian function to denoise the current frame image to obtain a smooth image;

[0100] The gradient magnitude and direction at each pixel point on the smoothed image are calculated using the first-order differential, and the gradient magnitude is non-maximum suppressed to obtain the gradient magnitude at each pixel point;

[0101] Compare the gradient magnitude at each pixel with the preset high threshold and the preset low threshold:

[0102] If the gradient amplitude at the pixel point is greater than a preset high threshold, the pixel point is determined to be an edge point;

[0103] If the gradient amplitude at the pixel point is less than the preset low threshold, it is determined that the pixel point is not an edge point;

[0104] If the gradient amplitude at the pixel point is not greater than the preset high threshold and not less than the preset low threshold, then determine whether there is an edge point among the adjacent pixels of the pixel point:

[0105] If there is an edge point, then the pixel point is determined to be an edge point;

[0106] If there is no edge point, it is determined that the pixel point is not an edge point;

[0107] All edge points are connected in sequence to obtain the initial edges of all power lines in the current frame image.

[0108] Specifically, in step S103-4, obtaining the target straight line segment includes:

[0109] In each combination, the longitudinal distance between any two straight line segments is calculated respectively;

[0110] Get the two straight line segments with the smallest longitudinal distance value, and compare the longitudinal distance values ​​of these two straight line segments with the preset longitudinal distance:

[0111] If the longitudinal distance between the two straight line segments is not less than the preset longitudinal distance, it is determined that the two straight line segments do not belong to the same power line;

[0112] If the longitudinal distance between the two straight line segments is less than the preset longitudinal distance, the Euclidean distances between the four endpoints of the two straight line segments are calculated, and the smallest Euclidean distance is compared with the preset transverse distance:

[0113] If the minimum Euclidean distance is not less than the preset horizontal distance, it is determined that the two straight line segments do not belong to the same electric line;

[0114] If the minimum Euclidean distance is less than the preset horizontal distance, it is determined that the two straight line segments belong to the same electric line; and the two end points with the minimum Euclidean distance are used as connection points to connect the two straight line segments to obtain the candidate straight line segments of the combination;

[0115] The longitudinal distance value and the Euclidean distance between the candidate straight line segment of the combination and any straight line segment in the remaining straight line segments in the combination are calculated, the straight line segments belonging to the same power line as the candidate straight line segment are connected, and the candidate straight line segments are updated until there is no straight line segment in the combination whose longitudinal distance value to the candidate straight line segment is less than the preset longitudinal distance, and the current candidate straight line segment is output as the target straight line segment of the combination.

[0116] The calculation of the longitudinal distance between any two straight line segments includes:

[0117] Calculate the shortest distances from the two endpoints of the first straight line segment to the straight line where the second straight line segment is located, and record them as the first distance and the second distance respectively;

[0118] Calculate the shortest distances from the two endpoints of the second straight line segment to the straight line where the first straight line segment is located, and record them as the third distance and the fourth distance respectively;

[0119] An average of the first distance, the second distance, the third distance, and the fourth distance is calculated as a longitudinal distance value between the first straight line segment and the second straight line segment.

[0120] Based on the extraction of edges by the Canny operator and the extraction of straight line segments by the Hough transform, the present invention proposes a line feature recovery method based on the grouping and merging of straight line segments. The identified straight line segments are divided into different groups according to the slope, and different groups represent different power lines. In each group, two straight line segments that meet the conditions are connected based on the longitudinal distance value and the Euclidean distance between the straight line segments, thereby obtaining the target edge of the power line represented by the combination. The present invention further splices the identified straight line segments based on the distance between the straight line segments, thereby improving the accuracy of power line edge recognition, thereby improving the accuracy of ground wire tracking during drone inspections.

[0121] Specifically, the candidate frame in the middle position is used as the target frame of the current frame image, including:

[0122] If the total number of candidate boxes If it is singular, the The candidate box is the target box of the current frame image; Indicates rounding up;

[0123] If the total number of candidate boxes If it is an even number, then the The candidate box is the target box of the current frame image.

[0124] Specifically, after obtaining the target frame of the current frame image, it also includes:

[0125] Calculate the intersection-and-union ratio of the target frame in the current frame image and the target frame in the previous frame image, and determine whether the intersection-and-union ratio is greater than zero:

[0126] If the intersection-and-union ratio is not greater than zero, the target frame in the previous frame image is used as the target frame in the current frame image;

[0127] If the IoU ratio is greater than zero, images of a preset number of frames before the current frame image are obtained, the IoU ratios between adjacent frame images are calculated and sorted, and the target frame in the image with the largest IoU ratio is used as the target frame in the current frame image.

[0128] Specifically, in step S105, the target frame of the current frame image is used as a reference, and the height difference between the center of the target frame of the current frame image and the center of the frame of the current frame image is calculated in real time to adjust the working angle of the drone, including:

[0129] If the height difference between the center of the target frame of the current frame image and the center of the frame of the current frame image is not greater than the preset height difference, the working angle of the drone gimbal is adjusted so that the center of the target frame of the current frame image is highly consistent with the center of the frame of the current frame image collected by the drone;

[0130] If the height difference between the center of the target frame of the current frame image and the center of the frame of the current frame image is greater than the preset height difference, the gimbal of the drone is returned to the center, and the drone is moved up and down by adjusting the flight control so that the center of the target frame of the current frame image is highly consistent with the center of the frame of the current frame image collected by the drone.

[0131] The 3D point cloud is obtained by collecting visible light images with high-precision point information for 3D reconstruction or laser radar scanning. Because the number of conductors in the tower lines of different voltage levels is different, a split conductor is used as an inspection unit. In this way, the inspection path of the ground wire can be determined by the connection point between the ground wire and the tower. Figure 3 As shown in the figure, it is a schematic diagram of the inspection path of the tower; the location of the connection point between the ground wire and the tower can be easily obtained through the three-dimensional point cloud. The upper part of the figure is two ground wires, and the lower part is three six-split conductors. There are five lines that need to be inspected in the inspection of the transmission line. The inspection of the tower in the embodiment of the present invention includes:

[0132] The drone starts from the connection point on one end of the lowest phase cross arm on one side of the tower, moves along the direction perpendicular to the cross arm to the connection point on the other end of the lowest phase cross arm, and detects the power lines within the range of the two connection points on the lowest phase cross arm of the tower;

[0133] After the detection of the lowest phase cross arm on one side of the tower is completed, the drone is ordered to move upward in a bow-shaped cruising path, and detect the power lines within the range of two connection points on the remaining phase cross arms on one side of the tower until the detection of the highest phase cross arm is completed;

[0134] At the uppermost horizontal arm, along the direction parallel to the uppermost horizontal arm, from one side of the tower, across the tower, to the other side of the tower;

[0135] On the other side of the tower, the drone starts from the connection point on one end of the uppermost horizontal arm on the other side of the tower, and moves toward the connection point on the other end of the uppermost horizontal arm in a direction perpendicular to the horizontal arm, and detects the power lines within the range of the two connection points on the uppermost horizontal arm of the tower;

[0136] After the inspection of the top phase cross arm on the other side of the tower is completed, the drone is ordered to move downward in a bow-shaped cruising path to inspect the power lines within the range of two connection points on the remaining phase cross arms on the other side of the tower until the inspection of the bottom phase cross arm is completed, completing the inspection of the tower.

[0137] The present invention divides the inspection into two parts: the pole tower and the ground wire. When inspecting the pole tower, each phase cross arm on both sides of the pole tower is inspected layer by layer in a bow-shaped patrol path, which can ensure that each layer of the cross arm and both sides of the pole tower are comprehensively inspected, and unnecessary round trips and repeated paths are reduced, thus saving inspection time and improving inspection efficiency.

[0138] Based on the above embodiment, this embodiment further calculates the minimum flight distance required for the drone to complete the inspection task by calculating the minimum flight distance required for the drone to complete the inspection task, so as to select a drone model that can meet the minimum flight time of the drone to perform the inspection task, including:

[0139] According to the distance between the starting tower and the destination tower, the number of ground wires between the starting tower and the destination tower, the preset inspection speed of the drone, and the time required for inspecting the tower, the minimum flight distance of the drone for the drone to complete the inspection of the starting tower and the inspection of the ground wire between the starting tower and the destination tower is obtained, which is expressed as: ;

[0140] in, Represents the distance between the starting tower and the destination tower, expressed as , , and They represent the horizontal, vertical and vertical coordinates of the tower in the spatial rectangular coordinate system respectively. represents the coordinates of the starting tower, Indicates the coordinates of the target tower; Indicates the number of ground wires between the starting tower and the destination tower. Indicates the time required for tower inspection. Indicates the preset inspection speed of the drone; Indicates the minimum flight distance of the drone;

[0141] Based on the minimum flight distance of the drone and the preset inspection speed of the drone, obtain the minimum flight time of the drone , expressed as: .

[0142] In overhead transmission lines, the suspended wires are flexible and will inevitably produce sag. Sag refers to the vertical distance from any point on the conductor suspension curve to the line connecting the two suspension points. Figure 4 As shown, it is a schematic diagram of sag; f is the maximum sag, that is, the vertical distance to the lowest point of the ground wire is the maximum sag.

[0143] In the embodiment of the present invention, when inspecting the ground wire, the RTK absolute position coordinates are used to record the flight altitude of the drone in real time. After completing the inspection of the ground wire, the maximum sag of the ground wire between the starting tower and the destination tower is calculated based on the flight path of the drone, which is expressed as:

[0144] ;

[0145] in, Indicates the maximum sag of the ground wire between the starting tower and the destination tower. Indicates the height from the connection point between the ground wire and the starting tower to the ground. Indicates the height from the connection point between the ground wire and the destination tower to the ground. Indicates the actual flight altitude of the drone at the midpoint between the starting tower and the destination tower.

[0146] In this embodiment, the drone is adjusted in real time along the ground wire. As long as the drone gimbal is centered, the trajectory of the drone during the entire flight process is consistent with the ground wire. By recording the position of the drone at different points, the curve of the ground wire can be fitted.

[0147] The core demand of ground wire inspection is to accurately check for various defects that may exist in the ground wire. Specifically, common ground wire defects include rust problems, which may cause the strength and conductivity of the ground wire to decrease; hanging foreign objects, such as kites, plastic sheets and other debris attached, will increase the extra weight of the ground wire and may cause short circuit risks; broken strands, which seriously weaken the mechanical bearing capacity of the ground wire; bulging conditions, which affect the normal shape and performance of the ground wire; and fault discharge points, which are often caused by damaged insulation or local abnormalities of the ground wire, which will threaten the safe and stable operation of the transmission line. This embodiment is based on the above embodiment. After completing the inspection of the ground wire, the present embodiment also includes: obtaining the current frame image collected during the drone inspection process, using a deep learning model to detect the current frame image, and identifying ground wire defects.

[0148] The ultra-ultra-high voltage line conductor inspection method based on unmanned aerial vehicle line inspection provided by the present invention performs defect recognition on the ground wire images collected during the inspection process to detect ground wire defects.

[0149] Based on the UAV-based line inspection method for ultra-high voltage line conductors provided by the present invention, an inspection system including a power grid private cloud, a data storage layer, a basic service layer, a business service layer and an application layer is constructed; Figure 5 The following is a schematic diagram of the inspection system structure; the specific functions of each layer are:

[0150] (1) Application layer: It mainly provides the functions of drone ground wire inspection and transmission line maintenance management, which includes the drone autonomous ground wire inspection APP and the transmission line ground wire maintenance management system.

[0151] (2) Business service layer: supports the implementation of inspection task management, image data processing, and hidden danger analysis functions in the application layer. It is composed of multiple REST-based services, including ground wire inspection task service, ground wire real-time identification and positioning service, drone autonomous navigation service, camera intelligent capture service, image management service, image storage service, defect hidden danger analysis service, and hidden danger management service.

[0152] (3) Basic service layer: It guarantees the stability and manageability of the entire system. It consists of functions such as registration center, service governance, configuration management, log collection, link monitoring, cluster monitoring and fault tolerance protection.

[0153] (4) Data storage layer: used to store various types of data in the system, such as image data, inspection records, and analysis results. MySQL, PostgreSQL, OSS (object storage service) and database resources, cache data clusters, etc. are used.

[0154] (5) Private cloud: responsible for data security and privacy. Mainly based on Rancher and k8s cloud services, it provides multiple elastic computing service instances, as well as confidential computer rooms and physical isolation services.

[0155] Specifically, the data storage in this embodiment mainly includes two storage methods: database and cloud storage. For example, MySQL and PostgreSQL relational databases are used to store and manage structured data; object storage services (OSS) are used to provide object storage functions for storing unstructured data. Rancher is used to manage k8s clusters to help users deploy and manage containerized applications; k8s is used to automatically deploy, expand and manage containerized applications.

[0156] Specifically, the inspection system constructed in this embodiment can be divided into two parts according to its function: a front-end drone autonomous ground wire inspection module and a ground wire inspection analysis and management module. Figure 6 The figure shows a functional architecture diagram of the inspection system.

[0157] Among them, when the ground wire is identified in the ground wire inspection module, the main steps of the Canny operator for edge detection are:

[0158] (1) Use Gaussian function to denoise and smooth the image;

[0159] (2) Calculate the gradient value and direction of each pixel through the first-order differential, and perform non-maximum suppression on the gradient amplitude to retain the local maximum; then, use the connected edges to obtain the final edge detection result;

[0160] (3) Use high threshold and low threshold to determine edge points; if the gradient amplitude of a pixel point is greater than the high threshold, it is considered an edge point; if it is less than the low threshold, it is considered not an edge point; if it is between the two, it is necessary to check whether the point is connected to a strong edge point. If so, it is considered an edge point, otherwise it is discarded.

[0161] However, using the Canny operator to extract edges and the Hough transform to extract straight line segments often results in fragmented and incomplete power line extraction. Therefore, this embodiment proposes a line feature recovery algorithm based on straight line segment grouping and merging to restore complete power lines. Figure 7 As shown in the figure, it is a complete power line edge restoration process flow chart, the specific steps are as follows:

[0162] (1) Use the slope of a straight line segment to determine whether it is parallel;

[0163] (2) Determine the longitudinal spacing between parallel straight line segments;

[0164] Reference Figure 8 The figure below is a schematic diagram of calculating the vertical spacing of straight line segments. The black line segments with endpoints are used to represent the power line segments in the aerial image. The two-point straight line equations of the two straight line segments are converted into general straight line equations. According to the point-to-straight line distance formula, the distance from point A to the straight line segment is obtained. The distance is , point B to the straight line segment The distance is , point C to the straight line segment The distance is , point D to the straight line segment The distance is .

[0165] Find the distance from the point to the line , , , After calculation, it is worth In practical applications, it is also necessary to find the straight line segment and other parallel straight line segments , , , … Distance , , … , ( For the straight line segment The number of parallel line segments). Compare , , , … The size of the vertical distance is the smallest. , then with the straight line segment The distance between The straight line can be considered to be In the aerial image it is the edge of the same power line, connecting the two.

[0166] (3) Determine the horizontal spacing between parallel straight line segments:

[0167] Determine whether the two straight line segments belong to the same power line edge. If so, connect them to restore the complete power line. By calculating the Euclidean distance between the four endpoints, find the two closest endpoints, assume that these two endpoints correspond to the damaged parts of the power line edge, and connect them. At this point, the complete power line edge is restored.

[0168] (4) Restore the complete power line edge:

[0169] When the horizontal and vertical distances of the extracted power line segments are too large, the straight line segments will be regarded as non-collinear straight line segments. Therefore, by setting the vertical and horizontal distance thresholds and comparing them with the minimum vertical and horizontal distances between the two straight line segments, it is determined whether they are collinear. If the vertical and horizontal distances are within the threshold range, the two power line segments are considered to be collinear, and they are fitted into the same straight line, thereby restoring the complete power line edge.

[0170] Specifically, in the embodiment of the present invention, the ground wire recognition is performed by using an algorithm, or based on a model, the ground wire recognition is performed by using an image segmentation method. In this embodiment, considering the front-end application, the model structure cannot be too complex, and the recognition speed cannot be too slow. Therefore, the network structure model of DeepLabV3+ is adopted, and MobileNet is selected as the backbone network. Fig. 9 As shown in the figure, it is a schematic diagram of the overall structure of DeepLabV3. Fig.10 As shown, it is a schematic diagram of the encoder-decoder structure of the DeepLabV3+ network; the DeeplabV3+ model of this embodiment adopts the mainstream encoder-decoder structure and uses MobileNet in the encoder process. Its computing speed is faster than Xception, the computing memory is smaller, the model size is also smaller, and it is more suitable for front-end applications.

[0171] RTK real-time dynamic positioning technology is a combined differential positioning system consisting of GPS measurement technology and data transmission technology. The base station sends its carrier observation and station coordinate information to the mobile terminal through the data link in real time. The mobile terminal receives the carrier phase of the GPS satellite and the carrier phase from the base station, and forms a phase differential observation value for real-time processing. It can give centimeter-level positioning results in real time. Its high-precision positioning allows the drone to accurately stay at the preset shooting point and automatically patrol and shoot standard images.

[0172] Based on the above embodiment, this embodiment optimizes the cruise path to determine the optimal path, taking into account the maximum flight time of the drone. , flight speed , the distance between the two towers , the number of conductors between the two towers , cross-tower time The route planning of the ground line inspection satisfies the following constraints:

[0173] The distance the drone can fly ( )satisfy: ;

[0174] The flight distance of the route planning mission meets the following requirements: ; The distance between the two towers ( ), can be calculated by GPS between the locations, using the Euclidean distance, that is: ; Where X, Y, H represent the position converted from longitude, latitude and altitude to plane coordinates.

[0175] At the same time, considering the safety of flight, a certain safety distance should be set from the connection point, which can be set to 7-15 meters. At the same time, a turning point should be set when entering the next wire and crossing the tower. When the UAV is flying, the nose of the drone should be facing the guide ground wire, the gimbal and the nose should be in the same direction, and it should fly sideways at a certain cruising speed.

[0176] The route planning path generated in this way can be imported into the UAV's autonomous inspection flight mission to assign flight missions to the UAV. In order to accurately capture the ground wire, it is necessary to use the onboard intelligent module to identify and track the ground wire to adjust the aircraft's attitude in real time.

[0177] Based on the above description, this embodiment extracts some features of the ground wire based on the algorithm of target detection, and compares the similarity of the target between different frames of the video to ensure that the same ground wire is extracted, thereby realizing ground wire tracking. However, when shooting multiple split wires at close range, it is difficult to identify them because some wires overlap to a high degree. To solve this problem, this embodiment adopts a strategy of simplifying the problem, and converts the ground wire tracking task into an operation of moving the ground wire to be tracked to the center of the picture. Specifically, the identified lines are merged into a frame from top to bottom, and then the frame in the middle position after the merger is selected and used as a reference for adjusting the flight attitude of the drone in the next step. By calculating the offset, the effect of adjusting the target ground wire to the center of the picture is continuously achieved. For example, when identifying six split wires, due to the high overlap due to the angle, 5 wires may be identified; in order to allow the six wires to be arranged in sequence in the picture, it is necessary to select one wire as the basis for adjustment, but the wire target similarity is very high, and it is difficult to confirm the center position of the dispersed selection. Select the intersecting parts of the target frame to merge, and select the middle position as the reference, so that the wires can be more evenly distributed in the picture, avoiding the situation where the ground wires are too close to the edge of the picture and part of the ground wires may exceed the range of the picture.

[0178] When adjusting the flight attitude of the drone based on the merged frame in the middle position, there are two methods: adjusting the gimbal and adjusting the flight control.

[0179] ① Adjust the gimbal: The autonomous route flight mission is set by the APP. After the front-end AI box is started, it identifies the ground wire in the picture, and uses the ground wire to control the gimbal according to the offset of the ground wire in the picture, so that the ground wire is moved to the center of the picture. At this time, the inspection is divided into two parts. The route flight is controlled by the APP, and the photo is taken by the airborne end. The control algorithm tends to be simple and the mission is clear. However, this method only adjusts the gimbal, which may cause the gimbal to exceed the rotation range due to excessive sag, resulting in the ground wire being unable to be adjusted to the center of the picture.

[0180] ② Adjust the flight control: This method first returns the gimbal to the center and aligns it horizontally with the ground wire, and then adjusts the flight control to move the drone up and down so that the ground wire is in the center of the screen. In this mode, both autonomous flight control and gimbal control are performed by the airborne end, and the control algorithm is relatively complex. Its advantage is that it can fit the ground wire curve and accurately calculate the sag at each point. However, this method is only suitable for scenes with an open environment around the power transmission line. If there are tree obstacles around, the drone will not be able to adjust up and down.

[0181] This embodiment can also use the above two methods in combination. In view of the larger rotation angle of the pan / tilt head can meet most situations, and the first method is simple and easy, which helps to quickly verify the effectiveness of ground wire identification, and the second method can calculate the sag in real time. The combination of the two can complement each other and improve the inspection effect and data collection quality.

[0182] The embodiment of the present invention loads the ultra-ultra-high voltage line conductor inspection method based on drone line inspection provided by the present invention on a drone to perform real-time identification of the ground wire of the transmission line. This embodiment is aimed at the DJI Matrice series multi-rotor drones, and selects airborne smart chips. The selected smart chips need to meet the following requirements: they can be carried on the drone, can interact with the drone, and the system environment can install the DJI Onboard SDK; they need to have a certain computing power to meet the calculation of the recognition algorithm, mainly the ground wire identification and tracking and sag calculation; the power supply must be below the maximum output voltage value of 24V of the drone power supply interface, and the lower the power consumption, the better; the price is cost-effective and the development cost is low.

[0183] Based on the above requirements, the mature edge computing chips available for selection include NVIDIA's Jetson TX2 and Huawei's Atlas200. The performance parameters of these two available chips are compared as shown in Table 1.

[0184] Table 1 Comparison of performance parameters of available chips

[0185] NVIDIA Jetson TX2 Atlas 200 size 87mm x 50 mm (core board size, without loading board) 137.8mm * 93.mm * 32.9mm AI computing power 1.3TFlops(FP16) 8TFlops / FP16,16TOPS / INT8 Memory 8G / 4G 8G / 4G Power consumption 7.5W 4GB: 5.5W / 8GB: 8W interface With a variety of different interface carriers x1 USB2.0 / USB3.0 weight 85g 30g Development Ecosystem Complete Weaker Model transplantation easy complex

[0186] Through testing, it was found that the Atlas200 algorithm model is difficult to transplant, which is not conducive to the development and testing of the algorithm; the JetsonTX2 interface is complete and mature, with many different types of carrier boards to choose from, and the algorithm transplantation is easy and convenient; therefore, this embodiment selects NVIDIA's Jetson TX2 as the onboard intelligent device.

[0187] Based on the above embodiments, the embodiments of the present invention can identify defects in the ground wire of the transmission line based on the images collected during the inspection process; by constructing a ground wire defect library, using the defect library to train different learning models, and by comparing the defect recognition accuracy shown by different models, the optimal model is selected and mounted on a drone to perform ground wire defect identification.

[0188] Since the number of initial ground wire samples is small, especially the label data of some categories (such as discharge burns, etc.) is less than 100, which greatly limits the adequacy and effectiveness of model training; therefore, an initial model can be built on the existing data set; with the help of this preliminary model, the original inspection pictures are screened to screen out pictures that may belong to specific defect categories, and then after manual review to confirm their accuracy, these newly screened valid pictures are added to the training library, and then the scale of training data is gradually expanded, the data diversity is enriched, the generalization ability and recognition accuracy of the model are improved, and a solid foundation is laid for more accurate detection of various defects of ground wires. And because the ground wire defects account for a small proportion of the picture, and the length-to-width ratios of the targets are very different, such as the length-to-width ratio of the ground wire rust is very large, while the length-to-width ratio of the burn and damage is small, so the images of the training set need to be adaptively cropped and appropriately augmented.

[0189] In the target detection task based on deep learning, since different category combinations lead to different recognition effects, it is particularly important to deeply analyze the sample space of the target and carefully compare different category combinations. On the basis of the constructed labeled sample library, different model combinations and different deep learning model training are selected, which will have different defect recognition accuracy. After independent training for different models, the recognition accuracy of each model is accurately evaluated. For the same category combination, different algorithms are used for comparative research, and an optimal method is selected and determined, and then continuous optimization and improvement are made on this basis to achieve the best model recognition effect.

[0190] Reference Fig.11 As shown in the figure, it is a flow chart of ground wire defect model training and recognition; multiple deep learning models (RetinaNet, Grid R-CNN, Cascade R-CNN, R-FCN, Faster R-CNN) are constructed, and more than 1,000 images (including non-defective images) are extracted for accuracy comparison; refer to Fig.12 As shown in the figure, it is a schematic diagram of the recognition accuracy comparison of different algorithms, among which the Faster R-CNN model has the best performance.

[0191] Improvements were made to Faster R-CNN, including adding feature pyramids (FPN) to better handle target features of different scales, using deformable convolution (DCN) to enhance the adaptability of convolutional neural networks to changes in object shape, adding online hard example mining (OHEM) during the training phase to focus on samples that are difficult to distinguish and improve the model's learning ability, and using the Soft-NMS method during testing to optimize the non-maximum suppression process and reduce the problem of repeated detection in target detection. At the same time, the existing samples were expanded using augmentation, and the accuracy was also improved to a certain extent. The accuracy of the final defect model is shown in Table 2:

[0192] Table 2 Ground wire defect identification accuracy table

[0193] Ground wire defect type Recall Accuracy AP Ground wire damage 0.423 0.592 0.442 Ground wire corrosion 0.664 0.607 0.696 Ground wire loose strands 0.341 0.448 0.339 Ground wire broken 0.243 0.467 0.298

[0194] For the research on defect recognition, due to the lack of samples, the accuracy of the model still needs to be further improved by continuously accumulating materials. Among the existing samples, the number of ground wire corrosion samples is large, and the trained model has the best accuracy. Broken strands have a certain effect, but due to the large difference in features, the recognition is also poor. Fig.13 The following is the effect diagram of defect recognition.

[0195] Based on the above embodiments, in this embodiment, the ultra-high voltage line conductor inspection method based on drone line inspection provided by the present invention and the selected onboard chip are used to perform field inspection, specifically including:

[0196] ① Route planning for the conductor and ground wire: import the collected point cloud data, select the start and end points of the conductor and ground wire inspection, select the spacer points in turn, automatically generate the conductor, ground wire and spacer points (the spacer automatically generates 6 photo points at 45° and 90° on both sides), and export the route adapted to the DJI Phantom and Mavic 3;

[0197] ② Positioning and navigation: Check the safety of the drone, load the route and take off; the drone automatically flies to the starting position of the ground wire route and turns on the automatic line patrol mode; the APP has a built-in ground wire recognition algorithm, which can control the up and down movement of the drone by real-time identification and calculation of the relative position of the ground wire and the drone; the drone performs fully autonomous flight operations according to the planned route; after completing the route, it automatically returns to land and ends the flight.

[0198] Reference Fig.14The figure shows a schematic diagram of ground wire identification. Compared with traditional manually controlled drones, the work efficiency of this embodiment is improved by about 30%, and the safety guarantee is improved by 100%. The fully autonomous inspection of drones further improves the efficiency by an additional 25%-30%, making flight operations more organized, simplified and intelligent, which effectively promotes the standardization and informatization of inspection work and achieves a goal orientation with safe and autonomous inspections as the core. Small drones are portable and quickly deployed, and can perform autonomous inspection tasks quickly and efficiently, greatly improving the efficiency of inspections. At present, inspection drones can adapt to AI inspection needs without the need to purchase additional equipment. While reducing human resource investment, it has successfully reduced operating costs by 30%. The lightweight ground wire AI inspection technology not only meets the content of ground wire acceptance inspections, but also meets the requirements of daily ground wire inspections, and is more suitable for the promotion and application of power inspections.

[0199] The ultra-ultra-high voltage line conductor inspection method based on unmanned aerial vehicle line inspection of the present invention takes a ground wire as an inspection unit, distinguishes each power line based on the target edge of multiple power lines in the ground wire, and merges the power lines into a candidate frame in pairs according to the position order from top to bottom, and then takes the candidate frame in the middle position as the target frame; using the identified target frame as a reference, the unmanned aerial vehicle flies up and down during the inspection process, so that the target frame of the current frame image is located in the middle of the frame of the current frame image, even if there is sag, it can ensure that the flying height of the unmanned aerial vehicle is always consistent with the height of the ground wire, thereby realizing accurate positioning and tracking of the ground wire, improving the inspection accuracy and efficiency, and making the inspection work more autonomous, convenient and safe. Based on the extraction of edges by the Canny operator and the extraction of straight line segments by the Hough transform, the present invention proposes a line feature recovery method based on the grouping and merging of straight line segments. The identified straight line segments are divided into different groups according to the slope, and different groups represent different power lines. In each group, two straight line segments that meet the conditions are connected based on the longitudinal distance value and the Euclidean distance between the straight line segments, thereby obtaining the target edge of the power line represented by the combination. The present invention further splices the identified straight line segments based on the distance between the straight line segments, thereby improving the accuracy of power line edge recognition, and thereby improving the accuracy of ground wire tracking during drone inspection. The present invention divides the inspection into two parts: the tower and the ground wire. When inspecting the tower, each phase cross arm on both sides of the tower is inspected layer by layer in a bow-shaped cruising path, which can ensure that each layer of the cross arm and both sides of the tower are comprehensively inspected, and unnecessary round trips and repeated paths are reduced, saving inspection time and improving inspection efficiency.

[0200] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0201] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0202] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0203] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0204] Obviously, the above embodiments are merely examples for the purpose of clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the present invention.

Claims

1. A method for inspecting conductors of ultra-high voltage lines based on unmanned aerial vehicle line inspection, characterized in that: include: Take any two adjacent towers in the area to be inspected as the starting tower and the destination tower, obtain the coordinates of the starting tower and the destination tower, and use them as the inspection starting point and inspection destination point of the drone respectively; Based on a three-dimensional point cloud image including a starting tower, a target tower, and multiple ground wires between the starting tower and the target tower, the multiple ground wires are obtained, one of the ground wires is used as an inspection unit, and each ground wire includes multiple split power lines; Based on each frame of the video stream collected by the drone in real time, the target edge of each power line in each ground wire in the current frame image is identified to distinguish each power line; According to the positions of the multiple power lines from top to bottom in the current frame image, the detection frames corresponding to two adjacent power lines are merged into one candidate frame, and multiple candidate frames arranged from top to bottom in the current frame image are obtained; the candidate frame in the middle position is used as the target frame of the current frame image; At each frame of the video stream collected by the drone in real time, the target frame of the current frame is used as a reference to calculate the height difference between the center of the target frame of the current frame and the center of the frame of the current frame in real time. The working angle of the drone is adjusted to keep the center of the target frame of the current frame consistent with the center of the frame of the current frame, and inspect a ground wire corresponding to the current inspection unit.

2. The method for inspecting ultra-high voltage line conductors based on unmanned aerial vehicle line inspection according to claim 1 is characterized in that: The method of identifying the target edge of each power line in each ground wire in the current frame image based on each frame image of the video stream collected in real time by the drone and distinguishing each power line includes: For each frame of the video stream collected by the drone in real time, the Canny operator is used to extract the initial edges of all power lines in the current frame. Based on the initial edge of each power line, multiple straight line segments are extracted using Hough transform; Calculate the slope of each straight line segment and group the straight line segments with equal slopes into a group; In each combination, the straight line segments belonging to the same power line in each combination are merged based on the longitudinal distance values ​​and the Euclidean distance between different straight line segments; The merged target straight line segment in each combination is obtained as the target edge of the power line represented by each combination, and each power line in the current frame image is distinguished.

3. The method for inspecting ultra-high voltage line conductors based on unmanned aerial vehicle line inspection according to claim 2 is characterized in that: For each frame of the video stream collected by the drone in real time, the Canny operator is used to extract the initial edges of all power lines in the current frame, including: Use Gaussian function to denoise the current frame image to obtain a smooth image; The gradient magnitude and direction at each pixel point on the smoothed image are calculated using the first-order differential, and the gradient magnitude is non-maximum suppressed to obtain the gradient magnitude at each pixel point; Compare the gradient magnitude at each pixel with the preset high threshold and the preset low threshold: If the gradient amplitude at the pixel point is greater than a preset high threshold, the pixel point is determined to be an edge point; If the gradient amplitude at the pixel point is less than the preset low threshold, it is determined that the pixel point is not an edge point; If the gradient amplitude at the pixel point is not greater than the preset high threshold and not less than the preset low threshold, then determine whether there is an edge point among the adjacent pixels of the pixel point: If there is an edge point, then the pixel point is determined to be an edge point; If there is no edge point, it is determined that the pixel point is not an edge point; All edge points are connected in sequence to obtain the initial edges of all power lines in the current frame image.

4. The method for inspecting the conductor of an ultra-high voltage line based on unmanned aerial vehicle line inspection according to claim 2 is characterized in that: In each combination, based on the longitudinal distance values ​​and the Euclidean distances between different straight line segments, the straight line segments belonging to the same power line in each combination are merged, including: In each combination, the longitudinal distance between any two straight line segments is calculated respectively; Get the two straight line segments with the smallest longitudinal distance value, and compare the longitudinal distance values ​​of these two straight line segments with the preset longitudinal distance: If the longitudinal distance between the two straight line segments is not less than the preset longitudinal distance, it is determined that the two straight line segments do not belong to the same power line; If the longitudinal distance between the two straight line segments is less than the preset longitudinal distance, the Euclidean distances between the four endpoints of the two straight line segments are calculated, and the smallest Euclidean distance is compared with the preset transverse distance: If the minimum Euclidean distance is not less than the preset horizontal distance, it is determined that the two straight line segments do not belong to the same electric line; If the minimum Euclidean distance is less than the preset horizontal distance, it is determined that the two straight line segments belong to the same electric line; and the two end points with the minimum Euclidean distance are used as connection points to connect the two straight line segments to obtain the candidate straight line segments of the combination; The longitudinal distance value and the Euclidean distance between the candidate straight line segment of the combination and any straight line segment in the remaining straight line segments in the combination are calculated, the straight line segments belonging to the same power line as the candidate straight line segment are connected, and the candidate straight line segments are updated until there is no straight line segment in the combination whose longitudinal distance value to the candidate straight line segment is less than the preset longitudinal distance, and the current candidate straight line segment is output as the target straight line segment of the combination.

5. The method for inspecting the conductor of an ultra-high voltage line based on unmanned aerial vehicle line inspection according to claim 2 is characterized in that: Calculate the longitudinal distance between any two straight line segments, including: Calculate the shortest distances from the two endpoints of the first straight line segment to the straight line where the second straight line segment is located, and record them as the first distance and the second distance respectively; Calculate the shortest distances from the two endpoints of the second straight line segment to the straight line where the first straight line segment is located, and record them as the third distance and the fourth distance respectively; An average of the first distance, the second distance, the third distance, and the fourth distance is calculated as a longitudinal distance value between the first straight line segment and the second straight line segment.

6. The method for inspecting ultra-high voltage line conductors based on unmanned aerial vehicle line inspection according to claim 1 is characterized in that: The candidate box in the middle is used as the target box of the current frame image, including: If the total number of candidate boxes If it is singular, the The candidate box is the target box of the current frame image; Indicates rounding up; If the total number of candidate boxes If it is an even number, then the The candidate box is the target box of the current frame image.

7. The method for inspecting ultra-high voltage line conductors based on unmanned aerial vehicle line inspection according to claim 1 is characterized in that: After obtaining the target frame of the current frame image, it also includes: Calculate the intersection-and-union ratio of the target frame in the current frame image and the target frame in the previous frame image, and determine whether the intersection-and-union ratio is greater than zero: If the intersection-and-union ratio is not greater than zero, the target frame in the previous frame image is used as the target frame in the current frame image; If the IoU ratio is greater than zero, images of a preset number of frames before the current frame image are obtained, the IoU ratios between adjacent frame images are calculated and sorted, and the target frame in the image with the largest IoU ratio is used as the target frame in the current frame image.

8. The method for inspecting the conductor of an ultra-high voltage line based on unmanned aerial vehicle line inspection according to claim 1 is characterized in that: Using the target frame of the current frame image as a reference, calculate the height difference between the center of the target frame of the current frame image and the center of the frame of the current frame image in real time to adjust the working angle of the drone, including: If the height difference between the center of the target frame of the current frame image and the center of the frame of the current frame image is not greater than the preset height difference, the working angle of the drone gimbal is adjusted so that the center of the target frame of the current frame image is highly consistent with the center of the frame of the current frame image collected by the drone; If the height difference between the center of the target frame of the current frame image and the center of the frame of the current frame image is greater than the preset height difference, the gimbal of the drone is returned to the center, and the drone is moved up and down by adjusting the flight control so that the center of the target frame of the current frame image is highly consistent with the center of the frame of the current frame image collected by the drone.

9. The method for inspecting the conductor of an ultra-high voltage line based on unmanned aerial vehicle line inspection according to claim 1 is characterized in that: After completing the inspection of the ground wire, the maximum sag of the ground wire between the starting tower and the destination tower is calculated based on the flight path of the drone, which is expressed as: ； in, Indicates the maximum sag of the ground wire between the starting tower and the destination tower. Indicates the height from the connection point between the ground wire and the starting tower to the ground. Indicates the height from the connection point between the ground wire and the destination tower to the ground. Indicates the actual flight altitude of the drone at the midpoint between the starting tower and the destination tower.

10. The method for inspecting the conductor of an ultra-high voltage line based on unmanned aerial vehicle line inspection according to claim 1, characterized in that: After completing the inspection of the ground wire, it also includes: obtaining the current frame image collected during the drone inspection process, using a deep learning model to detect the current frame image, and identifying ground wire defects.

11. The method for inspecting the conductor of an ultra-high voltage line based on unmanned aerial vehicle line inspection according to claim 1, characterized in that: Inspection of towers, including: The drone starts from the connection point on one end of the lowest phase cross arm on one side of the tower, moves along the direction perpendicular to the cross arm to the connection point on the other end of the lowest phase cross arm, and detects the power lines within the range of the two connection points on the lowest phase cross arm of the tower; After the detection of the lowest phase cross arm on one side of the tower is completed, the drone is ordered to move upward in a bow-shaped cruising path, and detect the power lines within the range of two connection points on the remaining phase cross arms on one side of the tower until the detection of the highest phase cross arm is completed; At the uppermost horizontal arm, along the direction parallel to the uppermost horizontal arm, from one side of the tower, across the tower, to the other side of the tower; On the other side of the tower, the drone starts from the connection point on one end of the uppermost horizontal arm on the other side of the tower, and moves toward the connection point on the other end of the uppermost horizontal arm in a direction perpendicular to the horizontal arm, and detects the power lines within the range of the two connection points on the uppermost horizontal arm of the tower; After the inspection of the top phase cross arm on the other side of the tower is completed, the drone is ordered to move downward in a bow-shaped cruising path to inspect the power lines within the range of two connection points on the remaining phase cross arms on the other side of the tower until the inspection of the bottom phase cross arm is completed, completing the inspection of the tower.

12. The method for inspecting the conductor of an ultra-high voltage line based on unmanned aerial vehicle line inspection according to claim 1, characterized in that: Also includes: According to the distance between the starting tower and the destination tower, the number of ground wires between the starting tower and the destination tower, the preset inspection speed of the drone, and the time required for inspecting the tower, the minimum flight distance of the drone for the drone to complete the inspection of the starting tower and the inspection of the ground wire between the starting tower and the destination tower is obtained, which is expressed as: ; in, Represents the distance between the starting tower and the destination tower, expressed as , , and They represent the horizontal, vertical and vertical coordinates of the tower in the spatial rectangular coordinate system respectively. represents the coordinates of the starting tower, Indicates the coordinates of the target tower; Indicates the number of ground wires between the starting tower and the destination tower. Indicates the time required for tower inspection. Indicates the preset inspection speed of the drone; Indicates the minimum flight distance of the drone; Based on the minimum flight distance of the drone and the preset inspection speed of the drone, obtain the minimum flight time of the drone , expressed as: .

Images