A method for monitoring and warning the tilt and posture of power line towers

Abstract

The present invention relates to the field of photogrammetry, and in particular to a method for monitoring and warning the inclination and posture of a power line tower, comprising: obtaining a first identical corner point pair and a second identical corner point pair in a perspective image, wherein the inclination error of the first identical corner point pair is less than the inclination error of the second identical corner point pair. For two adjacent perspective images F1 and F2, an area S where the first identical corner point pair in F2 is located is obtained on F1, and the second identical corner point pair in area S on F1 is corrected to obtain a corrected corner point pair, wherein the difference between the inclination error of the corrected corner point pair and the inclination error of the first identical corner point pair on F1 is less than a preset threshold; the inclination angle under F1 is re-obtained according to the first identical corner point pair on F1 and the corrected corner point pair; the inclination angle of the tower is obtained according to the inclination angle under all perspective images and a warning is issued. The present invention avoids the interference caused by uneven illumination and improves the accuracy of tower inclination measurement.

Description

Technical Field

[0001] The invention relates to the technical field of photogrammetry, and in particular to a method for monitoring and early warning the inclination and posture of a power line tower. Background Art

[0002] Power line towers are a key component of the power transmission system, and their stability directly affects the safety and reliability of power supply. The unstable foundation of power line towers, soil loss caused by natural phenomena such as river scouring and wind erosion, and natural disasters such as strong winds, floods, and earthquakes can all cause power line towers to tilt. If the problem of tower tilt is not discovered and handled in time, it may lead to tower collapse, line breakage, and even cause large-scale power outages.

[0003] The main method for detecting the tilt of power line towers is to use drones for inspections. The drones collect tower images and use photogrammetry technology to measure the tilt angles of the towers at different viewing angles. The tower posture is then detected over a long period of time to provide timely warnings for possible dangerous situations (such as situations where the tilt angle is large, or where the tilt angle changes significantly in a short period of time).

[0004] However, when using drones to inspect power line towers, weather and terrain reasons (such as clouds, fog, mountains and trees blocking light) result in the tower images collected by drones being unable to detect the structural features of all key positions on the towers, especially when measuring the overall tilt of the towers from a long distance. This causes errors in the tilt angles obtained by photogrammetry, which is particularly unfavorable for discovering smaller posture changes of the towers in a short period of time, and is not conducive to danger warnings, such as warnings of dangerous posture changes of towers after strong winds or earthquakes. Summary of the invention

[0005] In order to solve the above problems, the present invention provides a method for monitoring and early warning of the inclination and posture of a power line tower.

[0006] A method for monitoring and early warning the tilt and posture of a power line tower of the present invention adopts the following technical solutions:

[0007] An embodiment of the present invention provides a method for monitoring and early warning of the inclination and posture of a power line tower, the method comprising the following steps:

[0008] The images collected at different angles when the drone flies over the tower are recorded as viewpoint images;

[0009] After performing histogram equalization processing on each view image, each view image is matched to the reference image by using the corner point matching method to obtain a matched image. For several identical corner point pairs on the matching image and the reference image, the tilt angle under each view image is obtained according to the angle between the straight lines where the identical corner point pairs are located, and the tilt error of each identical corner point pair is obtained;

[0010] The plurality of identical corner point pairs in each viewing angle image include a first identical corner point pair and a second identical corner point pair, and a tilt error of the first identical corner point pair is smaller than a tilt error of the second identical corner point pair;

[0011] Among any two adjacent perspective images, the image collected when the distance to the tower is farthest is recorded as F1, and the image collected when the distance to the tower is closest is recorded as F2;

[0012] Obtaining the region S where the first identical corner point pair in F2 is located on F1, performing histogram equalization processing on the region S, and correcting the second identical corner point pair in the region S on F1 according to the first identical corner point pair in F2 to obtain a corrected corner point pair, wherein the difference between the tilt error of the corrected corner point pair and the tilt error of the first identical corner point pair on F1 is less than a preset threshold; and re-obtaining the tilt angle under F1 according to the first identical corner point pair on F1 and the corrected corner point pair;

[0013] The tower posture includes the tilt angles under all viewing angles. The tilt angle of the tower is obtained according to the tilt angles under all viewing angles and an early warning is issued.

[0014] Preferably, the step of obtaining the tilt angle under each viewing angle image according to the angle of the straight line where the same corner point pair is located, and obtaining the tilt error of each same corner point pair, comprises the following specific steps:

[0015] For any identical corner point pair P=(A, B) on the matching image and the reference image, A represents the corner point on the matching image, B represents the corner point on the reference image, and A and B are matching corner points;

[0016] Any other identical corner point pair other than the identical corner point pair P is recorded as a reference corner point pair, recorded as P1=(A1, B1), where A1 represents the corner point on the matching image, B1 represents the corner point on the reference image, and A1 and B1 are mutually matching corner points;

[0017] Get the angle between the straight line where A and A1 are located and the straight line where B and B1 are located, and record it as the inclination angle of any identical corner point pair P relative to each reference corner point pair;

[0018] For all reference corner point pairs other than the same corner point pair P, the average of the inclinations of the same corner point pair P relative to all reference corner point pairs is recorded as the observed inclination of the same corner point pair P;

[0019] The mean of the observed inclination angles of all pairs of identical corner points is recorded as the inclination angle under each viewing angle image;

[0020] The difference between the observed tilt angle of each identical corner point pair and the tilt angle under each viewing angle image is recorded as the tilt error of each identical corner point pair.

[0021] Preferably, the specific steps of the first identical corner point pair and the second identical corner point pair are as follows:

[0022] The same corner point pair whose tilt error is less than th1 is recorded as the first same corner point pair, and the same corner point pair whose tilt error is greater than or equal to th1 is recorded as the second same corner point pair, and th1 is the first preset threshold.

[0023] Preferably, the step of obtaining the region S on F1 where the first identical corner point pair in F2 is located comprises the following specific steps:

[0024] The corner point matching algorithm is used to obtain the corner point on F1 that matches the first identical corner point pair in F2, which is recorded as the target corner point, and the area formed by the neighborhood of all target corner points is recorded as area S.

[0025] Preferably, the method of correcting the second identical corner point pair in the region S on F1 according to the first identical corner point pair in F2 to obtain the corrected corner point pair comprises the following specific steps:

[0026] S1: Perform corner point detection on the pixels in area S on F1, and record the obtained corner points as initial correction corner points. Match the initial correction corner points with the corner points in the reference image to obtain several initial correction corner point pairs. The first identical corner point pair on F1 and the identical corner points in the initial correction corner point pair are recorded as updated corner point pairs. The first identical corner point pair and the second identical corner point pair are re-obtained according to the updated corner point pairs. The obtained first identical corner point pair and the second identical corner point pair are respectively recorded as identical corner point pair Q1 and identical corner point pair Q2. The difference between the comprehensive error of identical corner point pair Q2 and the comprehensive error of identical corner point pair Q1 is recorded as a. According to a, adjust the value of th1 lower to obtain the updated th1. According to the updated th1, re-obtain the first identical corner point pair in F2. Re-obtain area S on F1 where the first identical corner point pair in F2 is located, and perform histogram equalization on area S.

[0027] S2: Repeat S1 until a is less than or equal to the preset threshold. When a is less than or equal to the preset threshold, the obtained identical corner point pair Q2 is the corrected corner point pair.

[0028] Preferably, the comprehensive error of the same corner point pair Q2 is equal to the average value of all tilt errors in the same corner point pair Q2; the comprehensive error of the same corner point pair Q1 is equal to the average value of all tilt errors in the same corner point pair Q1.

[0029] Preferably, the value of th1 is adjusted down according to a to obtain updated th1, and the specific formula included is:

[0030] The updated value of th1 is represented by tt, tt=(1-a / 90)×th1.

[0031] Preferably, the specific steps of obtaining the same corner point pairs on the matching image and the reference image are as follows:

[0032] The SIFT corner point detection algorithm is used to obtain all corner points on the view image and the reference image, and the normalized cross-correlation matching algorithm is used to match the corner points in the view image with the corner points in the reference image to obtain several mutually matching corner point pairs. The mutually matching corner point pairs are used to affine transform the view image to obtain a matching image, and the several mutually matching corner point pairs are used as several identical corner point pairs on the matching image and the reference image.

[0033] Preferably, the tower inclination angle is the maximum value of the inclination angles under all viewing angle images.

[0034] Preferably, the early warning includes:

[0035] When the tower tilt angle of the pole tower is greater than the second preset threshold, or the difference in the tower tilt angles obtained during two consecutive drone inspections of the same pole tower is greater than the third preset threshold, an alarm is issued.

[0036] The beneficial effects of the technical solution of the present invention are:

[0037] In the present invention, the tilt error of the first identical corner point pair in each viewing angle image is less than the tilt error of the second identical corner point pair; the region S where the first identical corner point pair in F2 is located is obtained on F1, and after the region S is subjected to histogram equalization processing, the second identical corner point pair in the region S on F1 is corrected according to the first identical corner point pair in F2 to obtain a corrected corner point pair, and the difference between the tilt error of the corrected corner point pair and the tilt error of the first identical corner point pair on F1 is less than a preset threshold; the tilt angle under F1 is re-obtained according to the first identical corner point pair on F1 and the corrected corner point pair; this process is performed by using a viewing angle image (i.e., F2) that is closer to the tower. 2) obtains the local area S by using the pole tower structural features with smaller errors (that is, the first identical corner point pair in F2), so that the local area S contains the pole tower structural features with smaller errors. By re-equalizing S and correcting the pole tower structural features with larger errors in S in the perspective image far away from the pole tower (that is, F1) (that is, the second identical corner point pair in F1), the accuracy of obtaining the pole tower structural features in F1 is improved, and the problem of unclear or inaccurate pole tower structural features caused by uneven illumination is further avoided, thereby improving the accuracy of the inclination angle of F1 and reducing the measurement error of the pole tower inclination angle. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0039] Figure 1 A flowchart of a method for monitoring and early warning the tilt and posture of a power line tower provided by an embodiment of the present invention;

[0040] Figure 2 A schematic diagram of a predetermined trajectory of a drone during inspection provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0041] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of the specific implementation method, structure, features and effects of a power line tower tilt and posture monitoring and early warning method proposed by the present invention in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0042] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0043] The following is a detailed description of a method for monitoring and early warning of the inclination and posture of a power line tower provided by the present invention in conjunction with the accompanying drawings.

[0044] Embodiment 1:

[0045] See also Figure 1 , which shows a flowchart of a method for monitoring and early warning of the inclination and posture of a power line tower provided by Embodiment 1 of the present invention, the method comprising the following steps:

[0046] Step S101, images collected at different angles when the drone flies over the tower are recorded as view images; each view image is matched to a reference image by a corner point matching method to obtain a matching image.

[0047] The drone is equipped with a camera. In this embodiment, the camera captures grayscale images. In other embodiments, if the camera captures color images, they need to be grayscaled into grayscale images.

[0048] The drone is ordered to fly over each power line tower (hereinafter referred to as the tower) in turn and collect images. In this embodiment, the image of the tower is collected every 2 seconds.

[0049] When flying over each tower, the drone approaches a tower from far to near, circling while approaching, and then gradually moves away from the tower, and then approaches the next tower from far to near, also circling while approaching, and so on, after inspecting all the towers on the power line. The purpose of approaching and circling is that the drone can collect images of the tower from different perspectives by circling. The collected image of the tower is recorded as a perspective image, which contains the tower body of the entire tower.

[0050] In addition, the drone stores a baseline image of the tower, which refers to a clear image of the tower when it is not tilted.

[0051] After each viewing angle image is collected by the drone, the viewing angle image is preprocessed. The preprocessing in this embodiment includes performing histogram equalization processing on the entire viewing angle image. Its purpose is to take into account the uneven lighting caused by weather and terrain (such as clouds, fog, mountains and trees, etc. blocking the optical fiber), and the uneven brightness distribution of the viewing angle image, which in turn causes the structural features of the tower in the viewing angle image (such as corner features, edge features) to be unclear. For example, it is impossible to accurately distinguish the angle of the truss structure on the tower body, the length of the steel beam, etc. The structural features of the tower in the viewing angle image can be enhanced to a certain extent through histogram equalization.

[0052] In other embodiments, other preprocessing may be performed, such as using Gaussian filtering to remove noise in the perspective image, and using a defogging algorithm to remove fog in the perspective image.

[0053] A SIFT corner point detection algorithm is used to obtain all corner points in the view image and the reference image, and then a corner point matching algorithm (such as a normalized cross-correlation matching algorithm) is used to match the corner points in the view image with the corner points in the reference image to obtain a number of mutually matching corner point pairs. These mutually matching corner point pairs are used to affine transform the view image into the image space where the reference image is located to obtain a matching image. This process is well known, for example, CN110728296B discloses this method, and this embodiment will not be described in detail.

[0054] The obtained mutually matching corner point pairs represent the same structural features on the matching image and the reference image, that is, these mutually matching corner points are the same corner point pairs.

[0055] It should be noted that the matching image is obtained by affine transformation of the view image, so the corner points on the view image correspond one-to-one to the corner points on the matching image.

[0056] Any pair of identical corner points is denoted as P=(A, B), where A represents the corner point on the matching image (or the viewing image), and B represents the corner point on the reference image. These two corner points represent the same structure on the viewing image and the reference image, that is, the structural feature at A on the matching image is the same as the structural feature at B on the reference image.

[0057] Step S102: for a plurality of identical corner point pairs on the matching image and the reference image, obtain the tilt angle in each viewing angle image according to the angle of the straight line where the identical corner point pairs are located, and obtain the tilt error of each identical corner point pair. The plurality of identical corner point pairs in each viewing angle image include a first identical corner point pair and a second identical corner point pair, and the tilt error of the first identical corner point pair is less than the tilt angle error of the second identical corner point pair.

[0058] When the tower is tilted, corner point A on the matching image (or the viewing image) is rotated relative to corner point B on the reference image. The rotation angle of corner point A relative to corner point B reflects the tilt angle of the tower under each viewing image, which is simply recorded as the tilt angle under each viewing image.

[0059] As an example, the method for obtaining the tilt angle under each viewing angle image includes:

[0060] Any other identical corner point pair other than the identical corner point pair P is recorded as a reference corner point pair, recorded as P1=(A1, B1), where A1 represents the corner point on the matching image, B1 represents the corner point on the reference image, and A1 and B1 are mutually matching corner points.

[0061] Obtain the straight line where A and A1 are located, record it as the first straight line, obtain the straight line where B and B1 are located, record it as the second straight line, and the angle between the first straight line and the second straight line is recorded as the inclination angle of any identical corner point pair P relative to each reference corner point pair; wherein the angle refers to the rotation angle when the first straight line is rotated to coincide with the second straight line.

[0062] For all reference corner point pairs other than the same corner point pair P, the average of the inclinations of the same corner point pair P relative to all reference corner point pairs is recorded as the observed inclination of the same corner point pair P.

[0063] The mean of the observed inclination angles of all pairs of identical corner points is recorded as the inclination angle under each viewing angle image.

[0064] It should be further explained that the observed inclination angle of each identical corner point to P obtained above has errors, which is mainly contributed by the preprocessing process in step S101. Specifically, although preprocessing is used in step S101 to avoid the problems of uneven lighting, uneven brightness distribution of the viewing image, and unclear structural features of the tower in the viewing image, there is still a problem of unclear structural features in the local area of ​​the viewing image. There are errors in the corner point acquisition results and corner point matching results in these local areas, and even wrong corner point matching results or missing corner points, resulting in errors in the observed inclination angle of the same corner point to P, and errors in the inclination angle under each viewing image.

[0065] The error in the observed inclination of the same corner point pair P is simply referred to as the inclination error of the same corner point pair.

[0066] As an example, a method for obtaining the tilt error of each identical corner point pair includes:

[0067] The absolute value of the difference between the observed tilt angle of each identical corner point pair and the tilt angle under each viewing angle image is recorded as the tilt error of each identical corner point pair.

[0068] For the same corner point pairs in each perspective image (that is, the matching image obtained by affine transformation of the perspective image and several same corner point pairs on the reference image), the same corner point pairs whose tilt error is less than the first preset threshold th1 are recorded as the first same corner point pairs, and the same corner point pairs whose tilt error is greater than or equal to th1 are recorded as the second same corner point pairs. This embodiment is described using th1=8 degrees as an example. In other embodiments, th1 can be set to other values, and this embodiment does not make specific limitations.

[0069] Step S103, the image collected when the distance from the pole tower is farthest between any two adjacent perspective images is recorded as F1, and the image collected when the distance from the pole tower is closest is recorded as F2; ​​obtain the area S where the first identical corner point pair in F2 is located on F1, and correct the second identical corner point pair in area S on F1 according to the first identical corner point pair in F2 to obtain a corrected corner point pair, the difference between the tilt error of the corrected corner point pair and the tilt error of the first identical corner point pair on F1 is less than a preset threshold, and the tilt angle under F1 is obtained again according to the first identical corner point pair on F1 and the corrected corner point pair.

[0070] In the above process, the tilt angle under each perspective image and the tilt error of each identical corner point pair are obtained. As the drone flies over the tower from far to near, the lighting effect in the perspective image is different at different distances and different perspectives, and the structural features such as corner points in the perspective image are also different. This embodiment corrects the error in the previous perspective image (that is, the tilt error and tilt angle error of the identical corner point pair) based on the perspective image under another perspective when the distance is closer.

[0071] (1) Among any two adjacent perspective images, the image collected when the distance from the pole tower is farthest is recorded as F1, and the image collected when the distance from the pole tower is closest is recorded as F2; ​​according to the method in step S102, the same corner point pairs in F1 and the tilt error of each same corner point pair are obtained, and the same corner point pairs in F2 and the tilt error of each same corner point pair are obtained. It should be noted that F1 and F2 have overlapping perspectives, and the overlap of perspectives can be adjusted by setting the flight speed of the drone or the time interval for the drone's camera to collect images.

[0072] (2) Obtain the region S on F1 where the first identical corner point pair in F2 is located.

[0073] The tilt error of the first identical corner point pair in F2 is relatively small, and F1 and F2 are at adjacent viewing angles, and the influence of lighting is most similar. Therefore, area S contains structural features (or corner points) that can reduce the error. It is just that area S in F1 is affected by lighting, resulting in these structural features (or corner points) not being obtained, or not being obtained accurately.

[0074] As a preferred example, the method of obtaining the region S where the first identical corner point pair in F2 is located on F1 includes:

[0075] Corner point matching algorithm is used to match the corner points of F1 with the corner points in F2 to obtain several mutually matching corner point pairs. Any one of these mutually matching corner point pairs is denoted as P2=(A2, B2), wherein A2 represents the corner point in F1, B2 represents the corner point in F2, and corner points A2 and B2 match each other. In addition, F2 includes the first identical corner point pair (obtained in step S102). If corner point B2 is a corner point in the first identical corner point pair on F2, then A2 is denoted as the target corner point, and the area formed by the neighborhood of all target corner points is denoted as area S.

[0076] As an example, the area formed by the neighborhood of all target corner points is recorded as area S, including the following methods:

[0077] The N×N neighborhood of each target corner point is obtained (i.e., a square area with a side length of N and centered at the target corner point), and the area formed by the N×N neighborhoods of all target corner points is recorded as area S. This embodiment is described by taking N=7 as an example, and other embodiments may set N to other odd values.

[0078] As another example, the area formed by the neighborhood of all target corner points is recorded as area S, including the following methods:

[0079] According to the positions of the target corner points, all target corner points are clustered by mean shift to obtain all categories, and the convex hull area formed by all target corner points in each category is obtained. The convex hull area of ​​all categories is recorded as area S.

[0080] (3) Perform histogram equalization on region S.

[0081] The purpose is to consider the same corner point pairs and tilt angles obtained in the above steps. Due to the global histogram equalization processing of the entire view image (i.e. F1), the following problems are caused: there are still areas affected by illumination in the local area of ​​the view image, and the structural features that can reduce errors contained in these areas cannot be extracted (i.e., the corner points cannot be extracted, or the corner points can be extracted but the corner point pairs cannot be matched). This embodiment obtains the local area (i.e., area S) through the structural features that can reduce errors in F2 (i.e., the first same corner point pair) and performs equalization processing, which helps to avoid the above problems.

[0082] (4) After performing histogram equalization processing on region S, the second identical corner point pair in region S on F1 is corrected according to the first identical corner point pair in F2 to obtain a corrected corner point pair.

[0083] In this process, the second identical corner point pairs in F1 have a larger error, and the first identical corner point pairs in F2 have a smaller error. These second identical corner point pairs are corrected according to the first identical corner point pairs in F2, so that the errors of the corrected corner point pairs are reduced.

[0084] As an optional example, the second identical corner point pair in the region S on F1 is corrected according to the first identical corner point pair in F2 to obtain the corrected corner point pair, including the following method:

[0085] For the above-mentioned P2=(A2, B2), when A2 is the target corner point in region S and is also a corner point in the second pair of identical corner points on F1, A2 is retained. The corner points in the other second pairs of identical corner points on F1 are deleted.

[0086] In addition, for the area S after histogram equalization processing, corner point detection is performed on the pixels in the area S on F1 to obtain all corner points in the area S. These corner points and the retained corner points are re-matched with the corner points in the reference image, and the obtained matching corner point pairs are recorded as corrected corner point pairs.

[0087] As a preferred example, the second identical corner point pair in the region S on F1 is corrected according to the first identical corner point pair in F2 to obtain the corrected corner point pair, including the following method:

[0088] S1: For the area S after histogram equalization, perform corner point detection on the pixels in the area S on F1, and record the obtained corner points as initial correction corner points. The initial correction corner points are matched with the corner points in the reference image to obtain several initial correction corner point pairs;

[0089] Any one of the first identical corner point pair and the initial corrected corner point pair on F1 is recorded as an updated corner point pair, and the first identical corner point pair and the second identical corner point pair are re-obtained according to the updated corner point pair (similar to step S102), and the obtained first identical corner point pair and the second identical corner point pair are recorded as identical corner point pair Q1 and identical corner point pair Q2 respectively;

[0090] The average of all tilt errors of the same corner point pair Q2 is recorded as the comprehensive error of the same corner point pair Q2, and the average of all tilt errors of the same corner point pair Q1 is recorded as the comprehensive error of the same corner point pair Q1.

[0091] The absolute value of the difference between the comprehensive error of the same corner point pair Q2 and the comprehensive error of the same corner point pair Q1 is recorded as a; according to a, the value of th1 is lowered to obtain the updated th1.

[0092] The first identical corner point pair in F2 is obtained again according to the updated th1. The specific process is the same as step S102, that is, for the identical corner point pairs in F2, the identical corner point pairs whose tilt error is less than the updated th1 are recorded as the first identical corner point pairs, and the identical corner point pairs whose tilt error is greater than or equal to the updated th1 are recorded as the second identical corner point pairs.

[0093] Re-obtain the region S on F1 where the first identical corner point pair in F2 is located.

[0094] S2: Repeat the above step S1 until a is less than or equal to the preset threshold th (or stop repeating after repeating 5 times). The same corner point pair Q2 obtained at this time is the corrected corner point pair.

[0095] This embodiment is described by taking th=2 degrees as an example. In other embodiments, th may be set to other values, which are not specifically limited in this embodiment.

[0096] In this example, the region S is gradually changed by the difference a of the comprehensive error, so that the error of the corner points in the region S becomes smaller and smaller. In addition, the first and second identical corner point pairs (ie, Q1 and Q2) on F1 are also continuously updated during the above repeated execution process.

[0097] As an example, the value of th1 is adjusted down according to a to obtain the updated th1, including the calculation formula:

[0098] The updated value of th1 is represented by tt, tt=(1-a / 90)×th1. The larger a is, the smaller the updated th1 is, then the smaller the error of the first identical corner point pair retrieved in F2 is, and then the smaller the error of the structural features contained in the region S is, thus ensuring that the error of the corrected corner point is smaller.

[0099] (5) Re-obtain the tilt angle under F1 according to the first identical corner point pair on F1 and the corrected corner point pair (same as step S102). The tilt angle under F1 obtained here is more accurate and has a smaller error than the tilt angle obtained by step S102.

[0100] Step S104: Obtain the inclination angle of the tower according to the inclination angles under all viewing angle images and issue an early warning.

[0101] The tilt angles at all viewing angles describe the tower's posture.

[0102] Among the tilt angles under all viewing angle images, the one with the largest tilt angle is taken as the tilt angle of the pole tower. In other embodiments, the average of the tilt angles under all viewing angle images can be recorded as the tilt angle of the pole tower.

[0103] Furthermore, when the inclination angle of the tower is greater than a second preset threshold value th2, an early warning is issued. This embodiment is described by taking th2=10 degrees as an example.

[0104] Furthermore, all pole towers are inspected once at regular intervals (for example, one day), and the inclination angle of each pole tower is obtained. The absolute value of the difference between the inclination angles of the same pole tower during two adjacent inspections is obtained, and when the absolute value is greater than the third preset threshold th3, an early warning is also issued. This embodiment is described by taking th3=4 degrees as an example, and other embodiments may set th2 and th3 to other values, which are not specifically limited in this embodiment.

[0105] Embodiment 2:

[0106] For the drone used in the first embodiment, the inspection and shooting method of the drone specifically includes:

[0107] The drone flies along a pre-set trajectory. Figure 2 As shown, in other embodiments, the drone can be flown by remote control, passing through the pole towers to be inspected in turn, gradually approaching and circling each pole tower from a distance when flying over it, so that the image taken by the camera (i.e., the perspective image) includes the entire tower body of the pole tower, wherein the camera is installed on the drone through a gimbal, and when flying over each pole tower, the camera needs to face the tower body of the pole tower, and the specific method is:

[0108] After each view image is collected, the area of ​​the pole tower in the view image is obtained, the geometric center point of the area is obtained, and the rotation of the gimbal is controlled so that the geometric center point is at the center of the view image. At the same time, when the contour line of the area coincides with the boundary line of the view image, it means that the drone is close to the pole tower and the tower body is not completely in the view image. At this time, the drone is far away from the pole tower, so that the contour line of the area appears completely in the view image.

[0109] As an example, a method for obtaining the area of ​​the pole tower in the perspective image is: using the YOLOV4 neural network to obtain the bounding box of the pole tower, and the area where the bounding box is located is used as the area of ​​the pole tower in the perspective image.

[0110] As another example, a method for obtaining the area of ​​the pole tower in the perspective image is: using the DeepLabV3 neural network to obtain the semantic area of ​​the pole tower, and the semantic area is the area of ​​the pole tower in the perspective image.

[0111] It should also be noted that if the area of ​​the minimum circumscribed rectangle of the region of the pole tower in the perspective image is greater than 50% of the area of ​​the perspective image, all steps in the first embodiment are executed, otherwise no steps in the first embodiment are executed. In addition, in the present embodiment, if the area of ​​the region of the pole tower in the perspective image increases in two adjacent perspective images, it is considered that the pole tower is approaching, and if the area remains unchanged or decreases, it is considered that the pole tower is moving away. In the present embodiment, 50% can be set to other values, which are related to the shape of the pole tower and are not specifically limited in the present embodiment.

[0112] Embodiment three:

[0113] This embodiment provides a method for obtaining a reference image:

[0114] Use 3D design software such as AutoCAD and Maya to draw a 3D model of a non-tilted tower, and render the image of the 3D model at each viewing angle (azimuth angle in the bird's-eye view), which is recorded as a reference image. All reference images are stored on the drone. When the drone collects each viewing angle image, the corner points on the viewing angle image are matched with the corner points on the reference image to obtain all the matched corner point pairs.

[0115] It should be noted that, from the corner point detection algorithm (such as the SIFT corner point detection algorithm described in Example 1), it can be known that each corner point corresponds to a corner point descriptor (the corner point descriptor refers to a vector describing the corner point features), and the similarity (i.e., cosine similarity) of each matched pair of corner points is obtained according to the corner point descriptor, and the average of the similarities of all matched pairs of corner points is used as the similarity between the view image and the reference image. The reference image with the greatest similarity to each view image is used as the reference image corresponding to each view image.

[0116] As another example, the method for acquiring the reference image includes:

[0117] In advance, use a drone to fly around a non-tilted pole tower, record the angle of the drone (the azimuth of the drone in the bird's-eye view), take images of the pole tower at each angle, and use these images as reference images. After the drone captures each view image, divide the reference image into several 64×64 image blocks. If it is not evenly divided, fill the right and bottom boundaries of the reference image with several columns or rows of pixels with a grayscale value of 0 so that it can be just evenly divided. Each image block is used as a template, and template matching is performed on the view image to obtain the matching area of ​​the view image and the similarity between the template and the matching area. When all image blocks are used as templates, the average of the similarity between the template and the matching area is used as the similarity between the view image and the reference image. The reference image with the greatest similarity to each view image is used as the reference image corresponding to each view image. Template matching is a well-known technology, and this embodiment will not be described in detail.

[0118] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for monitoring and early warning of the inclination and posture of a power line tower, characterized in that: The method comprises the following steps: The images collected at different angles when the drone flies over the tower are recorded as viewpoint images; After performing histogram equalization processing on each view image, each view image is matched to the reference image by using the corner point matching method to obtain a matched image. For several identical corner point pairs on the matching image and the reference image, the tilt angle under each view image is obtained according to the angle between the straight lines where the identical corner point pairs are located, and the tilt error of each identical corner point pair is obtained; The plurality of identical corner point pairs in each viewing angle image include a first identical corner point pair and a second identical corner point pair, and a tilt error of the first identical corner point pair is smaller than a tilt error of the second identical corner point pair; Among any two adjacent perspective images, the image collected when the distance to the tower is farthest is recorded as F1, and the image collected when the distance to the tower is closest is recorded as F2; Obtaining the region S where the first identical corner point pair in F2 is located on F1, performing histogram equalization processing on the region S, and correcting the second identical corner point pair in the region S on F1 according to the first identical corner point pair in F2 to obtain a corrected corner point pair, wherein the difference between the tilt error of the corrected corner point pair and the tilt error of the first identical corner point pair on F1 is less than a preset threshold; and re-obtaining the tilt angle under F1 according to the first identical corner point pair on F1 and the corrected corner point pair; The tower posture includes the tilt angles under all viewing angles. The tilt angle of the tower is obtained according to the tilt angles under all viewing angles and an early warning is issued.

2. A method for monitoring and early warning the tilt and posture of a power line tower according to claim 1, characterized in that: The method of obtaining the tilt angle under each viewing angle image according to the angle of the straight line where the same corner point pair is located, and obtaining the tilt error of each same corner point pair, includes the following specific steps: For any identical corner point pair P=(A, B) on the matching image and the reference image, A represents the corner point on the matching image, B represents the corner point on the reference image, and A and B are matching corner points; Any other identical corner point pair other than the identical corner point pair P is recorded as a reference corner point pair, recorded as P1=(A1, B1), where A1 represents the corner point on the matching image, B1 represents the corner point on the reference image, and A1 and B1 are mutually matching corner points; Get the angle between the straight line where A and A1 are located and the straight line where B and B1 are located, and record it as the inclination angle of any identical corner point pair P relative to each reference corner point pair; For all reference corner point pairs other than the same corner point pair P, the average of the inclinations of the same corner point pair P relative to all reference corner point pairs is recorded as the observed inclination of the same corner point pair P; The mean of the observed inclination angles of all pairs of identical corner points is recorded as the inclination angle under each viewing angle image; The difference between the observed tilt angle of each identical corner point pair and the tilt angle under each viewing angle image is recorded as the tilt error of each identical corner point pair.

3. A method for monitoring and early warning the tilt and posture of a power line tower according to claim 1, characterized in that: The specific steps of the first identical corner point pair and the second identical corner point pair are as follows: The same corner point pair whose tilt error is less than th1 is recorded as the first same corner point pair, and the same corner point pair whose tilt error is greater than or equal to th1 is recorded as the second same corner point pair, and th1 is the first preset threshold.

4. A method for monitoring and early warning of the inclination and posture of a power line tower according to claim 1, characterized in that: The step of obtaining the region S on F1 where the first identical corner point pair in F2 is located includes the following specific steps: The corner point matching algorithm is used to obtain the corner point on F1 that matches the first identical corner point pair in F2, which is recorded as the target corner point, and the area formed by the neighborhood of all target corner points is recorded as area S.

5. A method for monitoring and early warning of the inclination and posture of a power line tower according to claim 3, characterized in that: The method of correcting the second identical corner point pair in the region S on F1 according to the first identical corner point pair in F2 to obtain the corrected corner point pair includes the following specific steps: S1: Perform corner point detection on the pixels in the area S on F1, and record the obtained corner points as initial correction corner points. Match the initial correction corner points with the corner points in the reference image to obtain several initial correction corner point pairs. The first identical corner point pair on F1 and the identical corner points in the initial correction corner point pair are recorded as updated corner point pairs. The first identical corner point pair and the second identical corner point pair are obtained based on the updated corner point pairs. The obtained first identical corner point pair and the second identical corner point pair are respectively recorded as identical corner point pair Q1 and identical corner point pair Q2. The difference between the comprehensive error of identical corner point pair Q2 and the comprehensive error of identical corner point pair Q1 is recorded as a. According to a, adjust the value of th1 lower to obtain the updated th1. According to the updated th1, obtain the first identical corner point pair in F2 again. Re-obtain the region S where the first identical corner point pair in F2 is located on F1, and perform histogram equalization processing on the region S; S2: Repeat S1 until a is less than or equal to the preset threshold. When a is less than or equal to the preset threshold, the obtained identical corner point pair Q2 is the corrected corner point pair.

6. A method for monitoring and early warning of the inclination and posture of a power line tower according to claim 5, characterized in that: The comprehensive error of the same corner point pair Q2 is equal to the average value of all tilt errors in the same corner point pair Q2; the comprehensive error of the same corner point pair Q1 is equal to the average value of all tilt errors in the same corner point pair Q1.

7. A method for monitoring and early warning of the inclination and posture of a power line tower according to claim 5, characterized in that: The method of adjusting the value of th1 to a lower value according to a to obtain updated th1 includes the following specific formula: The updated value of th1 is represented by tt, tt=(1-a / 90)×th1.

8. A method for monitoring and early warning of the inclination and posture of a power line tower according to claim 1, characterized in that: The specific steps of obtaining the same corner point pairs between the matching image and the reference image are as follows: The SIFT corner point detection algorithm is used to obtain all corner points on the view image and the reference image, and the normalized cross-correlation matching algorithm is used to match the corner points in the view image with the corner points in the reference image to obtain several mutually matching corner point pairs. The mutually matching corner point pairs are used to affine transform the view image to obtain a matching image, and the several mutually matching corner point pairs are used as several identical corner point pairs on the matching image and the reference image.

9. A method for monitoring and early warning of the inclination and posture of a power line tower according to claim 1, characterized in that: The tower inclination angle is the maximum value of the inclination angles under all viewing angle images.

10. A method for monitoring and early warning of the inclination and posture of a power line tower according to claim 1, characterized in that: The warnings include: When the tower tilt angle of the pole tower is greater than the second preset threshold, or the difference in the tower tilt angles obtained during two consecutive drone inspections of the same pole tower is greater than the third preset threshold, an alarm is issued.

Images