Method and device for calculating gradient of tower of ultrahigh-voltage power transmission line and medium

Abstract

The invention discloses an ultrahigh-voltage power transmission line tower gradient calculation method and device and a medium, and the method comprises the steps: fusing a high-definition camera image with a laser ranging eyepiece image, achieving the quick alignment of a heterogenous coordinate system through a rigid transformation matrix, and precisely capturing the depth information of the feature points of a tower target; a global-local detection model is constructed through cooperative work of a high-definition camera and a laser distance measuring sensor, and calculation complexity is reduced by adopting an SE-R3det network; the device and the medium can realize rapid and accurate tower inclination automatic measurement based on the method. The measurement efficiency is improved, laser ranging is adopted, feedback control is introduced, and automation of the whole process is achieved while the measurement precision is improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of power construction, and relates to a method and device for calculating the inclination of an ultrahigh-voltage power transmission line tower and a medium. BACKGROUND

[0002] As a key part of power transmission, the safety and stability of the power transmission process are closely related to the safe operation of the entire power system. As the core of the power transmission line, the power tower mainly supports and fixes the overhead line, adapts to various terrain conditions to realize long-distance crossing, lightning protection, insulation isolation and other important functions.

[0003] However, the power tower is affected by many factors in actual working conditions, and the inclination of the power tower may occur. The common inclination reasons can be summarized as follows.

[0004] 1. Natural environment influence: For example, strong typhoon weather, ice and snow storm, etc. The instantaneous impact generated by the extreme weather may cause structural deviation of the tower; acid rain weather causes corrosion to the tower structure material, which weakens the strength of the tower; poor soil conditions, ground loosening caused by earthquakes, landslides and mudslides in mountainous areas and some geological disasters affecting the tower foundation structure cause the tower to tilt.

[0005] 2. Construction materials and long-term load operation influence: The decline of the tower's bending and compression resistance caused by material aging such as cracking of the tower's foundation concrete, corrosion of steel, and the decline of the tower's stability caused by structural fatigue due to long-term load operation and environmental changes (such as wind, temperature, ice and snow coverage) and the tower structure bearing wire tension as the service life increases.

[0006] Therefore, regular detection of the tower inclination and timely maintenance measures can greatly avoid more serious power accidents and maintain the stable and efficient operation of the power system. At present, the common tower inclination detection mainly includes the following technical means:

[0007] 1. Manual inspection combined with professional instruments: The inspection personnel use professional instruments such as theodolites and total stations for on-site measurement. This measurement method is low in efficiency and is easily limited by terrain, and the applicable scenarios are limited.

[0008] 2. Fixed sensor detection: The inclination sensor and other devices are installed on the power tower to implement detection. Although this measurement method can realize the instant transmission of the inclination angle data, it has high installation and operation and maintenance costs, and the applicable scenarios are still limited.

[0009] 3. UAV inspection: through the UAV carrying laser radar or camera, the tilt angle is calculated through image processing means, this kind of inspection method needs powerful post image processing, and is obviously affected by weather.

[0010] 4. Laser scanning technology: three-dimensional point cloud data of the tower is obtained through laser beam, and the tilt angle is calculated by using the three-dimensional model, the detection method has high precision, but the actual cost is high, the professional level of technical personnel is required, and it is not suitable for large-scale ultra-high voltage transmission line project detection.

[0011] The patent application with publication number CN119036517A, "End effector guiding system and method based on 2D camera and laser range finder", can measure depth information, and can operate the end effector on planes of different heights and different inclinations to obtain the spatial information of the operation plane. However, the robot inspection needs path planning guidance, and the adaptability and accuracy are affected. SUMMARY

[0012] In order to overcome the defects of the prior art, the purpose of the present application is to provide a method, device and medium for calculating the inclination of an ultra-high voltage transmission line tower, which can accurately capture the depth information of the feature points of the tower target by fusing high-definition camera images and laser range finder images, and automatically and quickly measure the inclination of the tower by using heterogenous image acquisition tower feature detection. The non-automatic method for realizing tower inclination detection in the prior art, which relies too much on personnel operation of professional instruments and equipment, and pre-establishment of base stations, is solved, and rapid and accurate automatic measurement of tower inclination is realized.

[0013] To achieve the above purpose, the technical scheme adopted by the present application is as follows:

[0014] A method for measuring the inclination of an ultra-high voltage transmission line tower, comprising the following steps:

[0015] Step S1, initialization and preprocessing of the panoramic image of the power tower obtained by the high-definition camera and the local image of the power tower obtained by the laser ranging sensor eyepiece are respectively completed;

[0016] Step S2, the panoramic image of the power tower obtained by the high-definition camera is preprocessed, and the SE-R3det algorithm is used to realize global significant feature region detection of the tower;

[0017] Step S3, the local image of the power tower obtained by the laser ranging sensor eyepiece is preprocessed, and the SE-R3det algorithm is used to realize local significant feature region detection;

[0018] Step S4, according to the detected global and local salient feature regions, an initial matching corresponding set is obtained by using rigid transformation and structural similarity (SSIM) in a multi-scale space, target setting feature point fusion is realized, and a target setting point and a laser ranging sensor eyepiece position deviation are obtained;

[0019] Step S5, using local non-accurate map matching calculation, removing the error matching in the initial matching corresponding set, obtaining a precise matching corresponding set, and calculating a rigid transformation matrix of the power tower panoramic image and the power tower local image;

[0020] Step S6, in the power tower local image, taking the feature points in the local salient feature region in the precise matching corresponding set as a target setting point set, guiding the laser ranging sensor eyepiece to obtain the depth information of each target setting point;

[0021] Step S7, using the depth information of the N groups of target setting point sets and the image rigid transformation matrix, obtaining the depth information of the target setting point set in the power tower panoramic image, and calculating the inclination of the power tower.

[0022] The initialization and preprocessing operation in step S1 is specifically as follows:

[0023] Step S1.1, initializing the high-definition camera and the laser ranging sensor eyepiece, and the specific steps are as follows:

[0024] Step S1.1.1, hardware self-checking;

[0025] Step S1.1.2, calibration and calibration: self-defined setting of the image resolution and the acquisition frame rate of the high-definition camera, adjustment of the image exposure time, and adjustment of the color balance; using the calibration board to complete the calibration and calibration of the intrinsic parameters of the high-definition camera and the laser ranging sensor eyepiece, determining the emission frequency of the laser ranging sensor eyepiece and performing distance calibration; for the extrinsic parameters part, a multi-sensor joint extrinsic calibration is adopted, the high-definition camera and the laser ranging sensor eyepiece are rigidly fixed on the same calibration support, and a global coordinate system is established with the optical center of the high-definition camera as the origin; a composite calibration board containing three-dimensional features is used, feature point synchronous acquisition is performed, and a PnP (Perspective-n-Point) algorithm is used to solve the extrinsic matrix [R|t], wherein R is a rotation matrix and t is a translation vector;

[0026] Step S1.2, preprocessing the power tower panoramic image obtained by the high-definition camera, including denoising, sharpening enhancement, and image normalization;

[0027] Step S1.3, preprocessing the power tower local image obtained by the laser ranging sensor eyepiece, including denoising, gray optimization, image enhancement, and image normalization.

[0028] In the step S2, the power pole panoramic image obtained by the preprocessed high-definition camera is used to realize the inclined pole detection by using the SE-R3det network, and the target setting feature region detection is obtained, and the specific steps are as follows:

[0029] Step S2.1 image input and feature extraction

[0030] The panoramic image of the power pole obtained by the preprocessed high-definition camera is used as the network input, and the backbone part of the SE-R3det network is input, and a multi-scale feature map Fc is output, c represents different scale layers, and an SE module of the SE-R3det network is added after each layer of feature map for processing, and a corresponding channel description vector Z is generated by average pooling c , the weight vector ω is obtained by the full connection layer of the SE-R3det network c , and the multi-scale feature map Fc is weighted to obtain the enhanced feature map

[0031] Step S2.2 region proposal generation

[0032] Step S2.2.1 the enhanced feature map is input to the region proposal generation (RPN) network, and a series of candidate anchor frames (x, y, w, h, θ) are generated at each position of the enhanced feature map , wherein (x, y) is the center point coordinate, w and h are the width and height of the candidate anchor frame respectively, θ is the rotation angle, and the confidence of each candidate anchor frame is predicted, and a confidence threshold is set, and the candidate anchor frame below the confidence is directly discarded;

[0033] Step S2.2.2 further screen the candidate anchor frame: adopt the rotating non-maximum suppression (R-NMS) method, calculate the rotation intersection over union (R-IoU) and the angle difference Δθ = |θ i -θ j | of any two candidate anchor frames B i , B j , if R-IoU>τ IoU and Δθ<τ θ are met at the same time, it is judged as a redundant frame and is removed; the candidate anchor frame obtained after screening is combined with the characteristics of the power pole tower including the geometric shape and the structure layout, the candidate anchor frame angle in [80°, 100°] is retained, and the DBSCAN clustering is performed on the center coordinates of the candidate anchor frame to remove the isolated points; finally, N groups of candidate anchor frames are screened out, which represent the set of global salient feature regions of the tower, and are used for subsequent coarse matching.

[0034] The specific method of the step S3 is as follows:

[0035] Step S3.1 Image input and feature extraction

[0036] The pre-processed partial image of the electric power pole tower obtained through the laser ranging sensor eyepiece is input into the SE-R3det network, and the electric power pole tower partial enhanced feature map is obtained through SE module enhancement, average pooling and weighted processing

[0037] Step S3.2 Local salient feature region detection

[0038] Based on the enhanced feature map generated in step S3.1 The following operations are performed on its corresponding scale layer:

[0039] Step S3.2.1 Multi-scale anchor box setting and dynamic adjustment

[0040] Based on the prior knowledge of the local structure of the electric power pole tower, including the bolt spacing, angle steel width, the anchor box of n basic sizes and the rotation angle θ are predefined to adapt to different local salient feature regions of the electric power pole tower;

[0041] According to the gradient amplitude distribution of the enhanced feature map , the anchor box density is adaptively adjusted; if the gradient amplitude of a certain region exceeds the threshold value (such as gradient > 0.5), the anchor box is densely generated with α as the step size;

[0042] S3.2 Candidate region generation and optimization

[0043] The enhanced feature map is sent into the RPN network to generate candidate anchor boxes and predict confidence; according to the U-Net network segmentation, the electric power pole tower edge mask is calculated, the IoU value of the candidate anchor box and the edge mask is calculated, and the position accurate and high confidence candidate box is selected combined with the confidence; through non-maximum suppression (NMS), the redundant boxes are removed, and finally N groups of representative electric power pole tower local salient feature region sets are selected.

[0044] The specific method of the step S4 is that the target setting feature points of the panoramic image of the electric power pole tower and the partial image of the electric power pole tower are fused to obtain the position deviation of the target setting point and the laser ranging sensor, and the specific steps are as follows:

[0045] Step S4.1 Initial matching preparation in multi-scale space

[0046] The electric power pole tower panoramic image and the electric power pole tower partial image are respectively subjected to Gaussian downsampling to generate four-layer pyramids, in each pyramid, the SIFT algorithm is used to extract feature points from the global salient feature region and the local salient feature region obtained in steps S2 and S3, and a multi-dimensional descriptor vector is generated for each feature point; for the electric power pole tower panoramic image, the two-dimensional feature point coordinates detected by the SIFT algorithm are marked as The descriptor vector is denoted as For the power tower local image, the two-dimensional feature point coordinates detected by the SIFT algorithm are denoted as The descriptor vector is denoted as

[0047] Step S4.2. Rough matching based on rigid transformation and SSIM

[0048] Step S4.2.1, a nearest neighbor matching (KNN) algorithm is adopted to perform rough matching based on the two-dimensional feature points and the descriptor vectors detected by the SIFT algorithm: for each feature point in the power tower panoramic image, i.e., the two-dimensional feature point detected based on the SIFT algorithm, and the feature point descriptor vector set, the feature point in the power tower local image, i.e., the two-dimensional feature point detected based on the SIFT algorithm, and the feature point descriptor vector set, find the descriptor vector closest to the feature point in the Euclidean distance, to form a pair of rough matching point pairs, and the obtained rough matching point pairs are used to calculate the rigid transformation parameters;

[0049] Step S4.2.2, the preliminary alignment of the power tower panoramic image and the power tower local image candidate anchor frame is achieved by performing a rigid transformation on the power tower local image, including translation, rotation, and scaling operations, to obtain a modified local salient feature region;

[0050] Step S4.2.3, on each scale level of the four-layer pyramid obtained in step S4.1, the following operations are performed on the power tower local image after the rigid transformation and the power tower panoramic image:

[0051] Based on the candidate anchor frame as the basic region, the global salient feature region of the power tower panoramic image and the local salient feature region of the local image after the rigid transformation are Cartesian product paired to generate m x n groups of candidate region pairs, i.e., each global candidate anchor frame G i and each local candidate anchor frame L j form a unique matching pair;

[0052] In each matching pair, a fixed window size is selected, and the image blocks x and y are extracted by synchronous sliding, and the structural similarity (SSIM) is calculated; the calculation formula of the SSIM is as follows:

[0053]

[0054] Where μ x and μ y are window means, are variances, σ xy is a covariance, and C1 and C2 are constants;

[0055] The overall similarity of the candidate region is obtained by averaging all window SSIMs in the candidate box; the SSIM threshold τ is set, the candidate regions higher than the SSIM threshold are reserved, and the initial matching corresponding set M={(G i ,L′ j )|avg>τ}

[0056] Step S4.3 calculates the preliminary position deviation

[0057] For each matching pair in the initial matching corresponding set M, the panoramic candidate box G i center coordinates local candidate box L j 'center coordinates The position deviation is calculated: The deviation reflects the coordinate difference of the same physical structure region in the two images determined by the coarse matching; the 3δ principle is used to remove the obviously abnormal matching, and the mean μ Δx ,μ Δy and the standard deviation σ Δx ,σ Δy of all matching pairs are calculated, and the matching pairs satisfying |Δx-μ Δx |>3σ Δx or |Δy-μ Δy |>3σ Δy are removed; finally, N groups of matching regions with reasonable position deviation are reserved as the input of the accurate matching in step S5, providing the candidate matching region range.

[0058] The step S5 uses local non-accurate graph matching calculation to remove the error matching of feature points in the initial matching corresponding set, obtains the accurate matching corresponding point set, and calculates the rigid transformation matrix of the panoramic image of the power tower and the local image of the power tower. The specific steps are as follows:

[0059] Step S5.1 local non-accurate graph matching calculation

[0060] The Δx and Δy obtained in step S4.3 are used as the initial position constraint condition of the graph structure, and the local field graph is constructed; for each matching region (G i ,L′ j ) in the initial matching corresponding set, the internal feature point set P i , coordinate Q j , coordinate is extracted by using the SIFT algorithm respectively; two sets of feature points are modeled as graph structures G g and G l ; wherein the node is the feature point, and the edge is the Euclidean distance and the relative angle between the feature points; for example, a point p=(x l1 ,yl1 ) and a point q = (x g1 ,y g1 ) in the global salient feature region, the Euclidean distance is The relative angle difference is |θ p -θ q |, reflecting the consistency of the direction; the Hungarian Algorithm is used to solve the maximum weight matching of the graph structure, maximizing the topological consistency of the panorama and the local, and the error matching is filtered through the distance and angle constraints, and the optimal matching point set M

[0061] Step S5.2 calculates the rigid transformation matrix

[0062] Using the optimal matching point set M match obtained in step S5.1, the optimal transformation parameters in the rigid transformation matrix T are solved based on the least squares method, and the calculation formula is as follows:

[0063]

[0064] Where s is the scaling factor, θ is the rotation angle, and (t x ,t y ) is the translation amount;

[0065] This rigid transformation matrix T is used to convert the depth information obtained by the laser ranging sensor into three-dimensional coordinates in the panoramic image of the power tower, realizing the unification of the heterogeneous coordinate system; the m pairs of matching points with the highest confidence are retained as the target setting point set.

[0066] The feature points in the local salient feature region in the fine matching corresponding set are taken as the target setting point set, and the depth information of each target setting point is obtained by guiding the laser ranging sensor eyepiece, and the specific steps are as follows:

[0067] Step S6.1 guides the laser ranging sensor to align

[0068] In the local image region of the power tower, the feature points of the local salient region in the target setting point set finally obtained in step S5 are taken as the measurement target of the laser ranging sensor, and the center of the laser beam is controlled to align with the point subsequently. Using the intrinsic matrix K of the laser ranging sensor eyepiece calibrated in step S1.1.2, which contains the focal length f and the image principal point (u0′, v0′) information) and the extrinsic matrix [R|t], the two-dimensional coordinates (u, v) of the local image feature points are converted into three-dimensional coordinates in the local coordinate system, and the formula is as follows:

[0069]

[0070] Determination of the distance Δx', Δy', Δz' and the deflection angle θ that the laser ranging sensor center (X0, Y0, Z0) needs to move l , the formula is as follows:

[0071] Δx' = X target -X0

[0072] Δy' = Y target -Y0

[0073] Δz' = Z target -Z0

[0074]

[0075] The data processing unit generates a PWM pulse control signal, and through the encoder feedback current target position, compared with the target position, the PID algorithm is used to dynamically adjust the motor speed, forming a closed loop control, driving the motor on the laser ranging sensor to move; when the center of the laser beam and the pixel coordinate deviation of the target set point is less than 1 pixel, it is determined that the alignment is completed; the inertial measurement unit (IMU) is introduced to detect the vibration of the support, and when the severe vibration is detected, the measurement is suspended and the alarm is triggered, and the work is restored after the vibration is eliminated.

[0076] Step S6.2 depth information measurement and coordinate conversion

[0077] After the laser ranging sensor center is aligned with the target set point, a laser beam is emitted to the target set point through the time of flight principle (ToF), and the time difference Δt of laser emission and reception is recorded to obtain the depth information of the point, that is, the distance d from the target set point to the laser ranging sensor is calculated in combination with the speed of light c, wherein In combination with the sensor intrinsic matrix K, the two-dimensional coordinates of the aligned target point are converted into complete and accurate three-dimensional coordinates in the local coordinate system, and the formula is as follows:

[0078]

[0079] Step S6.3 exception handling and traversal control

[0080] It is judged whether the distance d is within the effective range, and the depth information is verified for effectiveness in combination with the signal-to-noise ratio (SNR) threshold, and the target point that meets the condition is recorded as an effective measurement target point, and the target point that does not meet the condition is transferred to the corresponding process of exception handling;

[0081] Reranging, return to step S4 for subsequent operation; if the ranging result d meets the effective range and signal-to-noise ratio threshold conditions, it is recorded as an effective measurement target point, and then the ranging is performed on the next target setting point in the local area; if the measurement is repeated multiple times and still does not meet the conditions, it is regarded as an abnormal point and is eliminated; all target points in the local area are traversed according to the spatial priority from left to right and from top to bottom; after the traversal ranging of all target setting points in the local area is completed, the next local significant area is entered, and step S4 is returned to select the target setting point for ranging.

[0082] The step S7 uses N groups of target setting points to rigidly transform the depth information and the image to obtain the target setting point pair depth information in the panoramic image of the power tower, and calculates the inclination of the power tower. The specific steps are as follows:

[0083] Step S7.1 Depth information mapping of panoramic image of power tower

[0084] According to the rigid transformation matrix T obtained in step S5, the distance d obtained in step S6 is mapped to the panoramic image coordinate system of the power tower based on the two-dimensional coordinates and the ranging result under the laser ranging sensor of the effective measurement target point in step S6; combined with the high-definition camera external parameter matrix [R|t] in step S1.1.2, the three-dimensional coordinates (X g ,Y g ,Z g ) containing panoramic depth information in the global coordinate system are obtained, and a target point three-dimensional point cloud set is generated as the basis data for inclination calculation;

[0085] Step S7.2 Three-dimensional coordinate point processing and tower center line vector calculation

[0086] On the basis of obtaining the three-dimensional point cloud set in step S7.1, the inclination of the tower is calculated through outlier elimination, confidence filtering and plane fitting, which is as follows:

[0087] RANSAC algorithm is used to eliminate outliers; plane fitting is performed on the three-dimensional point cloud set of the local area; three points are randomly selected to fit a plane as a candidate plane, and the residual of all points in the three-dimensional point cloud set of the current local area to the candidate plane is calculated; the candidate plane equation is aX+bY+cZ+d=0, and the residual is Set a residual threshold, and eliminate the points exceeding the threshold as outliers. The points meeting the threshold condition are judged as inliers; repeat the sampling N times, and select the candidate plane with the most inliers as the final fitting result; for each local area, a candidate plane and its corresponding inlier set are finally obtained for subsequent calculation;

[0088] For each candidate plane, the coordinates of the plane center point are calculated, and the geometric center coordinates O(x c ,yc ,z c ), O'(x c ', y c ', z c '), the connecting center point, obtaining the center line vector

[0089] Step S7.3 tower tilt calculation

[0090] According to the obtained center line vector of each candidate plane Respectively with the vertical line vector The angle alpha between the two vectors is calculated, and the calculation formula is: Convert radians to degrees, and the calculation formula is: That is the tower tilt angle theta.

[0091] According to the calculated tower tilt angle, the tilt degree in each dimension of the tower top, tower body and tower bottom can be divided, and the tower tilt condition can be judged.

[0092] A kind of tower tilt measurement device of ultrahigh voltage transmission line, comprising:

[0093] Depth camera, for obtaining two-way different sensor power tower image data, i.e. panoramic image of power tower obtained by high-definition camera and local image of power tower obtained by laser ranging sensor eyepiece, and completing initialization and preprocessing operation;

[0094] Memory, for storing computer program;

[0095] Processor, for performing the ultrahigh voltage transmission line tower tilt measurement method according to steps 1 to 7 when the computer program is executed, to measure tower tilt.

[0096] A computer readable storage medium, the computer readable storage medium stores computer program, the computer program is executed by processor when can be based on the ultrahigh voltage transmission line tower tilt measurement method according to steps 1 to 7, realize tower tilt measurement.

[0097] Compared with the prior art, the present application has the following beneficial effects:

[0098] 1. The step S1 of the present application fuses the information obtained by the high-definition camera and the laser ranging eyepiece image, realizes the rapid alignment of the heterogeneous coordinate system through the rigid transformation matrix, improves the efficiency of the sensor, adopts laser ranging and introduces feedback control, improves the measurement accuracy, and realizes the automation of the whole process.

[0099] 2. The application does not need to load fixed inclination sensor or laser radar on the tower, and avoids many restrictions such as being easily affected by weather and needing complex path planning in the process of unmanned aerial vehicle inspection. Through the cooperative work of high-definition camera and laser ranging sensor, steps S2 and S3 construct a global-local detection model, and SE-R3det network is used to reduce the calculation complexity.

[0100] 3. The application uses FPGA to accelerate feature extraction, at the same time, the rigid transformation matrix is updated in real time through steps S4-S5, the coordinate mapping accuracy is improved, the real-time transmission of inclination calculation is completed through the data communication module, and the monitoring effect is achieved.

[0101] 4. Compared with the traditional laser ranging which is only used for height measurement, the application introduces feedback control to dynamically guide the laser center to align the target point, realizes the dynamic adjustment of the laser ranging sensor pose, and improves the measurement accuracy of depth information.

[0102] Therefore, the application proposes a new way of combining high-definition camera and laser ranging sensor, realizes the detection of the real-time inclination angle of the tower based on multi-sensor fusion, and completes the detection of the real-time inclination angle of the tower based on SE-R3det algorithm. Significant breakthroughs have been achieved in measurement accuracy, application scenarios, operation and maintenance cost, real-time monitoring, etc. A new solution is provided for the inclination angle detection of the ultra-high voltage transmission line. Compared with single high-definition camera or laser radar application, the application constructs a "global-local" detection model, enhances the integrity of data acquisition through multi-dimensional data fusion, acquires three-dimensional space information, realizes the complementarity of multi-source data, and makes up for the limitations of single image analysis and processing. Compared with the expensive initial cost investment of 3D laser radar or fixed sensor, the combined scheme of the application has low operation and maintenance cost and does not need a large amount of manpower investment, can be applied to more complex scenes and cover a larger detection range. Compared with complex network models such as LSTM, the SE-R3det network significantly improves the adaptability and detection accuracy in complex environments, reduces the influence of environmental factors such as background reference and light through multi-layer feature extraction, and significantly reduces the calculation complexity, solving the severe demand for computing power of traditional methods. Compared with the selection method of traditional fixed feature points, the application uses SSIM structural similarity analysis combined with local inaccurate image matching, and then uses plane fitting residual to dynamically screen feature points, which has stronger robustness. Compared with the existing technology which depends on the initial position to divide the position, the application realizes accurate conversion of real-time coordinate system through the rigid transformation matrix, ensures the unity of the coordinate system under multiple perspectives, and realizes the integration of hardware devices through multi-scale feature matching, and realizes real-time online detection through the introduction of feedback mechanism. BRIEF DESCRIPTION OF DRAWINGS

[0103] Figure 1is a workflow diagram of the method of the present application.

[0104] Figure 2 is a tower measurement hardware arrangement diagram.

[0105] Figure 3 is a tower tilt calculation schematic diagram.

[0106] Figure 4 is a SE-R3det network structure diagram. DETAILED DESCRIPTION

[0107] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments of the present application.

[0108] According to Figure 1 The specific working process is given, and the method for measuring the tower tilt of an ultrahigh-voltage power transmission line according to the present application has the following specific steps:

[0109] Step S1, initialization and preprocessing operations of a panoramic image of a power tower obtained by a high-definition camera and a partial image of the power tower obtained by a laser ranging sensor eyepiece are respectively completed;

[0110] Step S1.1. Initialization, including the high-definition camera and the laser ranging sensor eyepiece, has the following specific steps:

[0111] Step S1.1.1. Hardware self-checking: checking whether the high-definition camera and the laser ranging sensor eyepiece are in a normal working state, the connection state of all hardware devices, and whether the high-definition camera and the laser ranging sensor eyepiece can normally communicate with the computer end;

[0112] Step S1.1.2. Calibration and calibration: customizing the image resolution and acquisition frame rate of the high-definition camera, adjusting the image exposure time, and adjusting the color balance; using a calibration board to complete the calibration and calibration of the intrinsic parameters of the high-definition camera and the laser ranging sensor eyepiece, determining the emission frequency of the laser ranging sensor eyepiece and performing distance calibration; for the extrinsic parameter part, a multi-sensor joint extrinsic parameter calibration is adopted, the high-definition camera and the laser ranging sensor are rigidly fixed on the same calibration support, and a global coordinate system is established with the optical center of the high-definition camera as the origin; a composite calibration board containing three-dimensional features (such as an orthogonal plane array or a spherical feature point) is used, feature points are synchronously collected, a PnP (Perspective-n-Point) algorithm is used to solve the extrinsic parameter matrix [R|t], where R is a rotation matrix and t is a translation vector; according to Figure 2The hardware arrangement fixes the high-definition camera and the laser ranging sensor eyepiece on the bracket, adjusts the position of the camera, and ensures that it can cover the tower body structure. After calibrating the parameters of the high-definition camera and the laser ranging sensor, the dual-source images are collected. Among them, the high-definition camera collects panoramic images of the tower, and the laser ranging sensor eyepiece collects local images of the tower. After collecting the panoramic images and local images, the collected images are preprocessed, mainly including image denoising, image enhancement, and image normalization.

[0113] Step S1.2. Preprocessing the high-definition camera image, including denoising, sharpening enhancement, and image normalization;

[0114] Denoising: A Gaussian filter algorithm is used to remove Gaussian noise from the image and optimize image edge details, providing high-quality input images for subsequent feature detection.

[0115] Sharpening enhancement: Laplace operator is used for sharpening operation to enhance image angle steel edge details, improve image contrast, and use histogram equalization method to uniform image gray scale distribution, making the tower structure details clearer.

[0116] Image normalization: Linear normalization method is used to map the pixel value of the image to the interval [0, 1], unify the scale of the image data, and facilitate subsequent SE-R3det feature extraction.

[0117] Step S1.3, preprocessing the laser ranging sensor eyepiece image, including denoising, gray optimization, image enhancement, and image normalization.

[0118] Denoising: Median filter algorithm is used to remove noise interference, and appropriate window size is selected according to image noise and detail preservation requirements to better reflect the true image features.

[0119] Gray optimization: Calculate the average gray value of the image, dynamically calculate the proportion factor α for linear adjustment, and enhance the target image and background contrast; cut off the gray optimized image to avoid gray value overflow;

[0120] Image enhancement: Adaptive histogram equalization (CLAHE) is combined with gray optimization to further refine the local contrast; limit the contrast amplification to avoid noise interference amplification; output the enhanced image to facilitate subsequent local significant feature region detection using SE-R3det algorithm;

[0121] Image normalization: Z-score method is used to realize local image normalization, unify the image quality of panoramic and local images, and facilitate subsequent image matching and feature detection work.

[0122] After preprocessing, noise interference can be eliminated and the tower structure (such as angle steel, bolts, etc.) can be highlighted, while normalization can ensure the fusion of heterogeneous images in subsequent steps.

[0123] Step S2: For the pre-processed panoramic image of the power tower acquired by the high-definition camera, the SE-R3det network is used to detect the global significant feature area of ​​the tower and obtain the target set feature area detection (for the specific network structure, see Figure 4 ), the specific steps are as follows: After the RPN network and R-NMS, the final candidate anchor frame is obtained. In this step, it is important to note that the feature layer selection of the SE-R3det network must conform to the structural characteristics of the tower. For example, the feature layer selection in S2.1 uses C5 to obtain overall contour features, C4 to obtain features of medium-sized components such as insulator strings, and C3 to obtain details at connection points. In practical applications, the SE-R3det network is computationally intensive and has certain requirements for hardware computing power. To meet the requirements of real-time and high efficiency, the SE-R3det network can be accelerated using FPGA.

[0124] Step S2.1 Image input and feature extraction

[0125] The local image G of the power tower obtained by the high-definition camera after preprocessing is used as the network input and input into the backbone of the SE-R3det network. The multi-scale feature map Fc is output, where c represents the number of different scale layers. The SE module of the SE-R3det network is added after each layer of feature map for processing, and the corresponding channel description vector Z is generated by average pooling. c , the weight vector ω is obtained through the fully connected layer of the SE-R3det network c , according to the weight, the multi-scale feature map Fc is weighted to obtain the enhanced feature map

[0126] Step S2.2 Region Proposal Generation

[0127] The enhanced feature map Input to the region proposal generation (RPN) network, after the enhanced feature map Generate a series of candidate anchor boxes (x, y, w, h, θ) for each position, where (x, y) is the center point coordinate, w, h is the width and height of the candidate anchor box, θ is the rotation angle, and predict the confidence for each candidate anchor box. Set a confidence threshold and discard the candidate anchor boxes below this confidence level.

[0128] Further screening of candidate anchor boxes: using the rotation non-maximum suppression (R-NMS) method, for any two candidate anchor boxes B i 、B j, calculate its rotation intersection over union (R-IoU) and angle difference Δθ = |θ i -θ j |, if R-IoU > τ IoU and Δθ < τ θ , it is determined as a redundant frame and is removed; the candidate anchor frames obtained after screening are combined with the characteristics of the power tower, including the geometric shape and structural layout, the candidate anchor frames with the angle in [80°, 100°] are retained, and isolated points are removed through DBSCAN clustering of the center coordinates of the candidate anchor frames; finally, N sets of candidate anchor frames representing the global salient feature region of the tower are screened out, which are used for subsequent coarse matching.

[0129] Step 3: The panoramic image collected after preprocessing is sent to the SE-R3det network (network structure is the same as step 2), and the candidate region frame is obtained. In this step, attention should be paid to setting multi-scale anchor frames and rotation angles according to local region features, and optimizing the candidate frame combined with the tower edge features. The global salient region should be uniformly distributed in the overall structure of the tower, which improves the accuracy of the later calculation of the inclination. If the deviation is too large, steps 1 and 2 should be performed for local adjustment. The Sobel operator is used to extract the edge gradient, and the U-Net output binary mask is used to ensure that the resolution is consistent with the local image.

[0130] Step 3 is specifically:

[0131] Step S3.2.1 Multi-scale anchor frame setting and dynamic adjustment

[0132] Based on the prior knowledge of the local structure of the power tower, including the bolt spacing and angle steel width, n basic size anchor frames and rotation angles θ are predefined to adapt to different local salient feature regions of the power tower.

[0133] According to the gradient amplitude distribution of the enhanced feature map , the anchor frame density is adaptively adjusted; if the gradient amplitude of a certain region exceeds the threshold (such as gradient > 0.5), anchor frames are densely generated with a step size of α.

[0134] S3.2 Candidate region generation and optimization

[0135] The enhanced feature map F c SE′ is sent to the RPN network to generate candidate anchor frames and predict the confidence; the power tower edge mask is obtained by U-Net network segmentation, the IoU value of the candidate anchor frame and the edge mask is calculated, and the candidate frame with accurate position and high confidence is selected combined with the confidence; through non-maximum suppression (NMS), overlapping redundant frames are removed, and finally N sets of candidate anchor frames representing the local salient feature region of the power tower are selected.

[0136] Step 4: Gaussian down-sampling is performed on the panoramic image and the local image respectively to construct a Gaussian pyramid to realize multi-scale space matching. SIFT is used to extract feature points and assign descriptors in the obtained candidate region box. The panoramic image feature point set and the local image feature point set are output. KNN algorithm is used for nearest zero matching to screen the matching point pairs. The obtained matching points are used to calculate the preliminary rigid transformation matrix. SSIM is used to calculate the structural similarity. The candidate regions with a structural similarity higher than the SSIM threshold are retained to obtain the initial matching corresponding set. The center coordinate deviation of the initial matching corresponding set is calculated to screen the matching region. The final matching region is used for subsequent operations in step S5. In the second step, the search range of the high layer is constrained by using the matching results of the low layer between the feature layers of different levels of the pyramid to improve the matching accuracy.

[0137] Step 4 is specifically:

[0138] Step S4.1 Initial matching preparation under multi-scale space

[0139] Gaussian down-sampling is performed on the panoramic image and the local image of the power tower respectively to generate a four-layer pyramid. In each pyramid layer, SIFT algorithm is used to extract feature points from the global and local significant feature regions obtained in steps S2 and S3, and a multi-dimensional descriptor vector is generated for each feature point. The two-dimensional feature point coordinates detected by the SIFT algorithm for the panoramic image of the power tower are denoted as The descriptor vector is denoted as For the local image of the power tower, the two-dimensional feature point coordinates detected by the SIFT algorithm are denoted as The descriptor vector is denoted as

[0140] Step S4.2 Coarse matching based on rigid transformation and SSIM

[0141] Step S4.2.1 uses the nearest neighbor matching (KNN) algorithm to perform coarse matching based on the two-dimensional feature points and descriptor vectors detected by the SIFT algorithm: for each feature point in the panoramic image of the power tower, i.e., the two-dimensional feature points detected by the SIFT algorithm, and the feature point descriptor vector set, the feature points in the local image of the power tower, i.e., the two-dimensional feature points detected by the SIFT algorithm, and the feature point descriptor vector set, find the nearest descriptor vector in the Euclidean distance, to form a pair of coarse matching points. The obtained coarse matching points are used to calculate the rigid transformation parameters;

[0142] Step S4.2.2 performs preliminary alignment of the candidate anchor boxes of the panoramic image of the power tower and the local image of the power tower by performing rigid transformation on the local image of the power tower, including translation, rotation, and scaling operations, to obtain the modified local significant feature region.

[0143] Step S4.2.3, for each scale level of the four-layer pyramid obtained in step S4.1, performs the following operation on the rigidly transformed power tower local image and the power tower panoramic image:

[0144] Based on the candidate anchor frame as the basic region, the global salient feature region of the power tower panoramic image and the local salient feature region of the rigidly transformed local image are Cartesian product paired to generate m x n groups of candidate region pairs, i.e., each global candidate anchor frame G i forms a unique matching pair with each local candidate anchor frame L j ′;

[0145] In each group of matching pairs, a fixed window size is selected, and image blocks x, y are synchronously slid and extracted, and the structural similarity (SSIM) is calculated. The calculation formula of SSIM is:

[0146]

[0147] Wherein μ x , μ y is the window mean, is the variance, σ xy is the covariance, and C1, C2 are constants;

[0148] The overall similarity of the candidate region is obtained by averaging the SSIM of all windows in the candidate frame. The SSIM threshold τ is set, and the candidate regions higher than the SSIM threshold are retained to construct the initial matching pair set M = {(G i ,L′ j )|avg>τ}

[0149] Step S4.3 calculates the preliminary position deviation

[0150] For each matching pair in the initial matching pair set M, the panoramic candidate frame G i center coordinates the local candidate frame L j ′ center coordinates The position deviation is calculated: The deviation is used to reflect the coordinate difference of the same physical structure region in the two images determined by coarse matching; the 3δ principle is used to remove significant abnormal matches, and the mean μ Δx , μ Δy and the standard deviation σ Δx , σ Δy of all matching pairs are calculated, and the matching pairs satisfying |Δx-μ Δx |>3σ Δx or |Δy-μ Δy |>3σ Δymatching pairs; finally, N groups of matching regions with reasonable position deviations are reserved as the input of the accurate matching in step S5, providing the range of candidate matching regions.

[0151] Step 5: The feature point set in the initial matching pair set M is modeled as a topological graph structure, where the nodes are the feature points and the edges are the Euclidean distances and relative angle differences between the feature points. The Hungarian algorithm is used to solve the maximum weight matching of the graph structure, and the final accurate matching point set is obtained by eliminating abnormal matching points with too large angle core distance differences. The obtained matching point set is used to optimize the rigid transformation parameters to obtain the final rigid transformation matrix T used for subsequent coordinate system conversion. In this step, the least squares method is used to solve the parameters in the rigid transformation matrix, and the matrix calculation is completed by FPGA. The matrix is optimized through multiple iterations. The residual error between all matching points is calculated, and the matrix parameters are adjusted according to the current error in each iteration, for example, using gradient descent or Levenberg-Marquardt algorithm. The residual error change is guaranteed to be ≤1e-5 to ensure that the obtained parameters are optimal.

[0152] Step 5 is specifically:

[0153] Step S5.1 Local Non-Accurate Graph Matching Calculation

[0154] The Δx and Δy obtained in step S4.3 are used as the initial position constraints of the graph structure to construct a local field graph; for each group of matching regions (G i ,L′ j ) in the initial matching pair set, the SIFT algorithm is used to extract the internal feature point set P i , coordinates Q j , coordinates The two groups of feature point sets are modeled as graph structures G g and G l , respectively; where the nodes are the feature points and the edges are the Euclidean distances and relative angles between the feature points; for example, a point p = (x l1 , y l1 ) in the local salient region and a point q = (x g1 , y g1 ) in the global salient feature region, the Euclidean distance is The relative angle difference is |θ p - θ q |, reflecting the consistency of the direction; the Hungarian algorithm is used to solve the maximum weight matching of the graph structure, maximizing the topological consistency of the panorama and the local, and filtering out the wrong matching points through distance and angle constraints, to generate the optimal matching point set

[0155] Step S5.2 calculates the rigid transformation matrix

[0156] Using the optimal matching point set M obtained in step S5.1 match , the optimal transformation parameters in the rigid transformation matrix T are solved based on the least square method, and the calculation formula is as follows:

[0157]

[0158] Where s is the scaling factor, θ is the rotation angle, (t x ,t y ) is the translation amount;

[0159] This rigid transformation matrix T is used to convert the depth information obtained by the laser ranging sensor into three-dimensional coordinates in the panoramic image of the power tower, realizing the unification of the heterogeneous coordinate system; the m pairs of matching points with the highest confidence are reserved as the target setting point set.

[0160] Step 6: According to the accurate matching point set obtained in step 5, the distance from the target setting point to the laser ranging sensor is calculated;

[0161] Step S6.1 guides the laser ranging sensor to aim at

[0162] In the local image area of the power tower, the feature points of the local salient region in the target setting point set finally obtained in step S5 are taken as the measurement target of the laser ranging sensor, and the center of the laser beam is controlled to aim at this point subsequently. Using the intrinsic matrix K of the laser ranging sensor eyepiece obtained in step S1.1.2, which contains the focal length f and the image principal point (u0′,v0′) information) and the extrinsic matrix [R|t], the two-dimensional coordinates (u,v) of the local image feature points are converted into three-dimensional coordinates in the local coordinate system, and the formula is as follows:

[0163]

[0164] The distance Δx', Δy', Δz' that the center (X0,Y0,Z0) of the laser ranging sensor needs to move and the deflection angle θ that the laser ranging sensor needs to rotate are determined l , and the formula is as follows:

[0165] Δx'=X target -X0

[0166] Δy'=Y target -Y0

[0167] Δz'=Z target -Z0

[0168]

[0169] The PWM pulse control signal is generated by the data processing unit, the current target position is fed back through the encoder, and the motor speed is dynamically adjusted after comparison with the target position by using the PID algorithm to form a closed-loop control to drive the motor mounted on the laser ranging sensor to move; when the pixel coordinate deviation between the center of the laser beam and the target set point is less than 1 pixel, it is determined that the alignment is completed; the inertial measurement unit (IMU) is introduced to detect the vibration of the support, and when severe vibration is detected (such as shaking caused by excessive wind speed), the measurement is suspended and an alarm is triggered, and the work is resumed after the vibration is eliminated.

[0170] Step S6.2 depth information measurement and coordinate conversion

[0171] After the center of the laser ranging sensor is aligned with the target set point, a laser beam is emitted to the target set point by the time-of-flight principle (ToF), and the time difference At between the emission and reception of the laser is recorded to obtain the depth information of the point, that is, the distance d from the target set point to the laser ranging sensor is calculated by combining the speed of light c, wherein, By combining the sensor intrinsic matrix K, the two-dimensional coordinates of the aligned target point are converted into complete and accurate three-dimensional coordinates in the local coordinate system, and the formula is as follows:

[0172]

[0173] Step S6.3 exception handling and traversal control

[0174] It is judged whether the distance d is within the effective range, and the depth information is verified for effectiveness in combination with the signal-to-noise ratio (SNR) threshold, and the target point that meets the condition is recorded as an effective measurement target point, and the target point that does not meet the condition is transferred to the corresponding process for exception handling;

[0175] The ranging is performed again, and the subsequent operation is returned to step S4; if the ranging result d meets the effective range and signal-to-noise ratio threshold conditions, it is recorded as an effective measurement target point, and then the next target set point in the local area is measured, and if the repeated measurement still does not meet the conditions, it is excluded as an abnormal point; all target points in the local area are traversed according to the spatial priority from left to right and from top to bottom; after the traversal and measurement of all target set points in the local area are completed, the next local significant area is entered, and the target set point is selected for measurement in step S4. The following will be explained in combination with Figure 2 . It is assumed that the first local area contains 4 feature points, which are stored in the point set queue according to the spatial distribution priority to obtain the preset order of traversal.

[0176] Taking one of them as an example, it is assumed that its coordinates in the laser ranging sensor eyepiece are (x L ,y L ), and the rotation matrix R and the translation vector t determined by the extrinsic parameters and the intrinsic focal length f are used to convert it to the global coordinate system (x G ,yG ), real-time detection of support vibration by IMU (inertial measurement unit) is introduced to improve system anti-interference capability.

[0177] According to the currently recorded central position coordinates of the laser ranging sensor, the distance Δx', Δy', Δz' that the center of the laser ranging sensor needs to move and the deflection angle θ that the center needs to rotate are determined l , and the movement process depends on the cooperation of the hardware part, which is specifically shown in Figure 2 . The data processing unit generates corresponding PWM pulse signals according to Δx', Δy', Δz' and θ l , drives the motor to move according to a certain step size, adjusts the control signal by feeding back the actual position of the current center of the laser ranging sensor, and introduces a PID module for fine adjustment.

[0178] After alignment, depth information measurement is performed according to S6.2. If the signal-to-noise ratio (SNR) is less than a certain threshold or the depth information of the point cannot be obtained, the sensor is fine-tuned again. If the depth information of the point cannot be obtained for three consecutive times, the point is determined to be invalid information, and the point is dynamically excluded. Return to the local image to reprocess the image. The above part is the specific processing steps for obtaining the depth information of the feature points in a local area.

[0179] The image coordinates of the next target point are extracted from the queue, and the three-dimensional coordinates are calculated to control the center of the laser ranging sensor to align with it again for ranging. After traversing all the target points in this area, the data storage is completed, and the next local area is jumped to for operation. Repeat the above operation until all the matching local areas in the global image complete the depth information measurement. According to the obtained depth information, the tower inclination is calculated.

[0180] Step 7: Convert the three-dimensional coordinates in the local coordinate system to the panoramic global coordinate system using the rigid transformation matrix T to calculate the tower inclination, as shown in Figure 3 ; select N sets of target set points, which should be evenly distributed in the overall structure of the tower to reflect the overall structure of the tower. Figure 3 , two groups of target points form two tower representative surfaces, and the center line coordinates of each surface are calculated according to the calculation formula given in step S7. The two coordinates are used to form a center line vector. The inclination angle is calculated and the tower inclination is judged according to the formula given in S7.3. In this step, the RANSAC algorithm is used to analyze the fitting residual of the three-dimensional point cloud set, and the plane normal vector is verified to ensure the consistency of the local plane structure. Whether the plane is valid is determined by calculating the included angle between the two plane normal vectors to adapt to the complex projection relationship of the non-overhead view angle.

[0181] Step 7 is specifically:

[0182] Step S7.1 power tower panoramic image depth information mapping

[0183] According to the rigid transformation matrix T obtained in step S5, the distance d obtained in step S6 is mapped to the power tower panoramic image coordinate system based on the two-dimensional coordinates and ranging results of the effective measurement target points under the laser ranging sensor in step S6; combined with the high-definition camera external parameter matrix [R|t] in step S1.1.2, the three-dimensional coordinates (X g ,Y g ,Z g ) containing panoramic depth information in the global coordinate system are obtained, and a target point three-dimensional point cloud set is generated as the basis data for inclination calculation;

[0184] Step S7.2 three-dimensional coordinate point processing and tower center line vector calculation

[0185] On the basis of obtaining the three-dimensional point cloud set in step S7.1, the tower inclination is calculated through outlier rejection, confidence filtering and plane fitting, as follows:

[0186] The RANSAC algorithm is used to reject outliers; the three-dimensional point cloud set of the local area is fitted with a plane; three points are randomly selected to fit a plane as a candidate plane, and the residual of all points in the three-dimensional point cloud set of the current local area to the candidate plane is calculated; the candidate plane equation is aX+bY+cZ+d=0, and the residual is A residual threshold is set, and points exceeding the threshold are considered outliers and are rejected, and points meeting the threshold condition are determined as inliers; repeat sampling N times, and select the candidate plane with the most inliers as the final fitting result; for each local area, a candidate plane and its corresponding inlier set are finally obtained for subsequent calculation;

[0187] For each candidate plane, the plane center point coordinates are calculated, and the geometric center coordinates O(x c ,y c ,z c ), O'(x c ′,y c ′,z c ′) of two candidate planes (such as the tower top and the tower body) are calculated, respectively, the center points are connected to obtain the center line vector

[0188] Step S7.3 tower inclination calculation

[0189] According to the obtained center line vectors of each candidate plane and the plumb line vector , the included angle a between the two vectors is calculated, and the calculation formula is: Convert radians to degrees, and the calculation formula is: , which is the tower inclination angle θ.

[0190] According to the calculated tower tilt angle, the tilt degree in each dimension of the tower top, tower body and tower bottom can be divided, and the tower tilt condition is judged.

[0191] The above description is only a more detailed description of the technical solutions of the present embodiment, and is not a limitation. Those skilled in the art should understand that the technical solutions can be modified, and the deficiencies can be improved and replaced. The modification should be within the technical scope of the present application.

Claims

1. A method for measuring the inclination of an ultra-high voltage transmission line tower, characterized in that: The steps are as follows: Step S1, respectively completing the initialization and preprocessing operations of the panoramic image of the power tower obtained by the high-definition camera and the local image of the power tower obtained by the eyepiece of the laser ranging sensor; Step S2: using the SE-R3det algorithm to detect the global significant feature areas of the power towers in the pre-processed panoramic image acquired by the high-definition camera; Step S3: Detecting local significant feature areas using the SE-R3det algorithm on the pre-processed local image of the power tower obtained by the eyepiece of the laser ranging sensor; Step S4: Based on the detected global and local salient feature regions, an initial matching correspondence set is completed in a multi-scale space using rigid transformation and structural similarity (SSIM), thereby achieving target setting feature point fusion and obtaining the position deviation between the target setting point and the eyepiece of the laser ranging sensor; Step S5: using local inexact graph matching calculation to remove erroneous matches in the initial matching correspondence set, obtaining a precise matching correspondence set, and calculating the rigid transformation matrix of the power tower panoramic image and the power tower local image; Step S6: In the local image of the power tower, the feature points in the local significant feature area of ​​the precise matching correspondence set are used as the target set point set, and the eyepiece of the laser ranging sensor is guided to obtain the depth information of each target set point; Step S7: using the N groups of target set point set depth information and the image rigid transformation matrix, obtain the target set point set depth information in the panoramic image of the power tower and calculate the inclination of the power tower.

2. The method for measuring the inclination of an ultra-high voltage transmission line tower according to claim 1, characterized in that: The initialization and preprocessing operations described in step S1 are specifically performed as follows: Step S1.1: Initialize the high-definition camera and the laser ranging sensor eyepiece. The specific steps are as follows: Step S1.1.1 Hardware self-test; Step S1.1.2 Calibration and calibration: Customize the image resolution and acquisition frame rate of the HD camera, adjust the image exposure time, and adjust the color balance; use the calibration plate to complete the calibration and calibration of the internal parameters of the HD camera and the laser ranging sensor eyepiece, determine the emission frequency of the laser ranging sensor eyepiece and perform distance calibration; for the external parameters, use multi-sensor combined external parameter calibration, rigidly fix the HD camera and laser ranging sensor on the same calibration bracket, and establish a global coordinate system with the optical center of the HD camera as the origin; use a composite calibration plate containing three-dimensional features, synchronously collect feature points, and use the PnP (Perspective-n-Point) algorithm to solve their external parameter matrices [R|t] respectively, where R is the rotation matrix and t is the translation vector; Step S1.2: pre-processing the panoramic image of the power tower obtained by the high-definition camera, including denoising, sharpening enhancement, and image normalization; Step S1.3: Preprocess the local image of the power tower obtained by the eyepiece of the laser ranging sensor, including denoising, grayscale optimization, image enhancement, and image normalization.

3. The method for measuring the inclination of an ultra-high voltage transmission line tower according to claim 1, characterized in that: In step S2, the pre-processed panoramic image of the power tower obtained by the high-definition camera is used to detect the tilted tower using the SE-R3det network to obtain the target set feature area detection. The specific steps are as follows: Step S2.1 Image input and feature extraction The panoramic image of the power tower obtained by the high-definition camera after preprocessing is used as the network input and input into the backbone of the SE-R3det network. The multi-scale feature map Fc is output, where c represents the number of different scale layers. The SE module of the SE-R3det network is added after each layer of feature map for processing, and the corresponding channel description vector Z is generated by average pooling. c , the weight vector ω is obtained through the fully connected layer of the SE-R3det network c , according to the weight, the multi-scale feature map Fc is weighted to obtain the enhanced feature map Step S2.2 Region Proposal Generation Step S2.2.1: Enhance the feature map Input to the region proposal generation (RPN) network, after the enhanced feature map Generate a series of candidate anchor boxes (x, y, w, h, θ) for each position, where (x, y) is the center point coordinate, w, h is the width and height of the candidate anchor box, θ is the rotation angle, and predict the confidence for each candidate anchor box. Set a confidence threshold and discard the candidate anchor boxes below this confidence level. Step S2.2.2 further screens the candidate anchor boxes: Use the rotation non-maximum suppression (R-NMS) method to select any two candidate anchor boxes B i 、B j , calculate its rotation intersection over union (R-IoU) and angle difference Δθ=|θ i -θ j |, if R-IoU>τ is satisfied at the same time IoU value and Δθ<τ θ The redundant frames are removed. The candidate anchor frames obtained after screening are combined with the characteristics of the power tower, including the geometric shape and structural layout, and the candidate anchor frames with angles between [80°, 100°] are retained. The center coordinates of the candidate anchor frames are clustered by DBSCAN to remove isolated points. Finally, N groups of candidate anchor frames are screened out to represent the set of global significant feature areas of the tower for subsequent coarse matching.

4. The method for measuring the inclination of an ultra-high voltage transmission line tower according to claim 1, wherein: The specific method of step S3 is: Step S3.1 Image input and feature extraction The local image of the power tower obtained by the pre-processed laser ranging sensor eyepiece is input into the SE-R3det network, and the local enhanced feature map of the power tower is obtained through SE module enhancement, average pooling and weighted processing. Step S3.2: Detection of local salient feature regions Based on the enhanced feature map generated in step S3.1 Perform the following operations on the corresponding scale layer: Step S3.2.1 Multi-scale anchor frame setting and dynamic adjustment Based on the prior knowledge of the local structure of the power tower, including bolt spacing, angle steel width, predefined anchor frame size n and rotation angle θ, it can adapt to different local significant feature areas of the power tower; According to the enhanced feature map Gradient amplitude distribution, adaptively adjust the anchor box density; If the gradient amplitude of a certain area exceeds the threshold, the anchor box is densely generated with a step size of α; S3.2 Candidate region generation and optimization The enhanced feature map The image is fed into the RPN network to generate candidate anchor boxes and predict their confidence. The edge mask of the power tower is obtained based on the U-Net network segmentation. The IoU value between the candidate anchor box and the edge mask is calculated. The candidate boxes with accurate position and high confidence are selected based on the confidence. Non-maximum suppression (NMS) is used to remove overlapping redundant boxes, and finally a set of N groups representing the local significant feature areas of the power tower is selected.

5. The method for measuring the inclination of an ultra-high voltage transmission line tower according to claim 1, characterized in that: The specific method of step S4 is: fusing the panoramic image of the power tower and the local image of the power tower to realize target setting feature point fusion, and obtaining the position deviation between the target setting point and the laser ranging sensor. The specific steps are as follows: Step S4.1 Initial matching preparation in multi-scale space Gaussian downsampling is performed on the panoramic image and the local image of the power tower to generate a four-layer pyramid. In each layer of the pyramid, the SIFT algorithm is used to extract feature points from the global and local salient feature regions obtained in steps S2 and S3, and a multi-dimensional descriptor vector is generated for each feature point. For the panoramic image of the power tower, the coordinates of the two-dimensional feature points detected by the SIFT algorithm are marked as The descriptor vector is recorded as For the local image of the power tower, the coordinates of the two-dimensional feature points detected by the SIFT algorithm are marked as The descriptor vector is recorded as Step S4.2: Rough matching based on rigid transformation and SSIM Step S4.2.1 uses the KNN algorithm to perform coarse matching based on the two-dimensional feature points and descriptor vectors detected by the SIFT algorithm: for each feature point in the panoramic image of the power tower, that is, the two-dimensional feature point detected by the SIFT algorithm and the set of feature point descriptor vectors, and for each feature point in the local image of the power tower, that is, the two-dimensional feature point detected by the SIFT algorithm and the set of feature point descriptor vectors, find the descriptor vector with the closest Euclidean distance to the feature point to form a pair of coarse matching point pairs, and use the obtained coarse matching point pairs to calculate the rigid transformation parameters; Step S4.2.2 performs a rigid transformation on the local image of the power tower, including translation, rotation, and scaling operations, to achieve preliminary alignment of the candidate anchor frames of the panoramic image of the power tower and the local image of the power tower, thereby obtaining a corrected local salient feature region; Step S4.2.3: At each scale level of the four-layer pyramid obtained in step S4.1, perform the following operations on the rigidly transformed local image of the power tower and the panoramic image of the power tower: Based on the candidate anchor frame as the base region, the global salient feature region of the power tower panoramic image is paired with the local salient feature region of the rigidly transformed local image by Cartesian product to generate m×n sets of candidate region pairs, that is, each global candidate anchor frame G i With each local candidate anchor box L j 'Form a unique matching pair; In each matching pair, a fixed window size is selected, and image blocks x and y are extracted by sliding synchronously to calculate their structural similarity (SSIM). The calculation formula of SSIM is: where μ x , μ y is the window mean, is the variance, σ xy is the covariance, C1, C2 are constants; The average SSIM of all windows in the candidate box is taken to obtain the overall similarity of the candidate area; the SSIM threshold τ is set by custom, and the candidate areas above the SSIM threshold are retained to construct the initial matching correspondence set M = {(G i ,L′ j )|avg>τ} Step S4.3 Calculate the preliminary position deviation For each set of matching pairs in the initial matching set M, record the panoramic candidate box G i Center coordinates Local candidate box L j ′Center coordinates Calculate its position deviation: The deviation is used to reflect the coordinate difference of the same physical structure region in the two images determined by rough matching; the 3δ principle is used to eliminate significant abnormal matches, and the mean μ of all matching pairs is calculated. Δx 、μ Δy , and the standard deviation σ Δx , σ Δy , eliminate the ones that satisfy |Δx-μ Δx |>3σ Δx or |Δy-μ Δy |>3σ Δy Finally, retain N groups of matching areas with reasonable position deviations as input for precise matching in step S5 to provide candidate matching area ranges.

6. The method for measuring the inclination of an ultra-high voltage transmission line tower according to claim 1, wherein: In step S5, local inexact graph matching calculation is used to remove erroneous matches existing in the feature points in the initial matching corresponding set, obtain a precise matching corresponding point set, and calculate the rigid transformation matrix of the power tower panoramic image and the power tower local image. The specific steps are as follows: Step S5.1 Local inexact graph matching calculation The Δx and Δy obtained in step S4.3 are used as the initial position constraints of the graph structure to construct a local domain graph; for each set of matching regions (G i ,L′ j ), respectively use SIFT algorithm to extract its internal feature point set P i ,coordinate Q j ,coordinate Model the two sets of feature points as graph structures G g and G l ; Among them, nodes are feature points, and edges are Euclidean distances and relative angles between feature points; for example, a point p in a local salient area is (x l1 ,y l1 ) and a point q=(x g1 ,y g1 ), then its Euclidean distance is The relative angle difference is |θ p -θ q |, reflecting its directional consistency; the Hungarian Algorithm is used to solve the maximum weight matching of the graph structure, maximize the panoramic and local topological consistency, and generate the optimal matching point set by filtering out incorrect matches under distance and angle constraints. Step S5.2 Calculate the rigid transformation matrix Using the optimal matching point set M obtained in step S5.1 match , based on the least squares method, the optimal transformation parameters in the rigid transformation matrix T are solved. The calculation formula is as follows: Where s is the scaling factor, θ is the rotation angle, (t x ,t y ) is the translation amount; This rigid transformation matrix T is used to convert the depth information obtained by the laser ranging sensor into three-dimensional coordinates in the panoramic image of the power tower, realizing the unification of heterogeneous coordinate systems; the m pairs of matching points with the highest confidence are retained as the target set point set.

7. The method for measuring the inclination of an ultra-high voltage transmission line tower according to claim 1, characterized in that: In step S6, in the local image of the power tower, the feature points in the local significant feature area of ​​the precise matching correspondence set are used as the target set point set, and the laser ranging sensor eyepiece is guided to obtain the depth information of each target set point. The specific steps are as follows: Step S6.1: Guide the laser ranging sensor to align In the local image area of ​​the power tower, the characteristic point of the local significant area of ​​the target set point obtained in step S5 is used as the measurement center of the laser ranging sensor, and the center of the laser beam is subsequently controlled to align with this point; using the intrinsic parameter matrix K of the laser ranging sensor eyepiece calibrated in step S1.1.2, which contains the focal length f and the image principal point (u0′, v0′) information) and the extrinsic parameter matrix [R|t], the two-dimensional coordinates (u, v) of the local image feature point are converted to three-dimensional coordinates in the local coordinate system. The formula is as follows: Determine the distance Δx', Δy', Δz' that the center of the laser ranging sensor (X0, Y0, Z0) needs to move and the deflection angle θ that needs to be rotated l , the formula is as follows: Δx'=X target -X0 Δy'=Y target -Y0 Δz'=Z target -Z0 The data processing unit generates a PWM pulse control signal, and the encoder feeds back the current target position. After comparing it with the target position, the PID algorithm is used to dynamically adjust the motor speed, forming a closed-loop control to drive the movement of the motor mounted on the laser ranging sensor. When the pixel coordinate deviation between the center of the laser beam and the target set point is less than 1 pixel, the alignment is determined to be complete. An inertial measurement unit (IMU) is introduced to detect bracket vibration. When severe vibration is detected, the measurement is suspended and an alarm is triggered. The operation is resumed after the vibration is eliminated. Step S6.2 Depth information measurement and coordinate conversion After the center of the laser ranging sensor is aligned with the target set point, a laser beam is emitted to the target set point through the time of flight principle (ToF). The time difference Δt between laser emission and reception is recorded to obtain the depth information of the point. That is, the distance d from the target set point to the laser ranging sensor is calculated in combination with the speed of light c. Combined with the sensor intrinsic parameter matrix K, the two-dimensional coordinates of the aligned target point are converted into complete and accurate three-dimensional coordinates in the local coordinate system. The formula is as follows: Step S6.3 Exception handling and traversal control Determine whether the distance d is within the valid range and verify the validity of the depth information by combining the signal-to-noise ratio (SNR) threshold. If the conditions are met, it will be recorded as a valid measurement target point. If the conditions are not met, the process will be transferred to the corresponding abnormal handling process. Re-measure the distance and return to step S4 for subsequent operations. If the distance measurement result d meets the effective range and signal-to-noise ratio threshold conditions, it is recorded as a valid measurement target point, and then the next target set point in the local area is measured for distance. If the conditions are still not met after repeated measurements, it is removed as an abnormal point. All target points in the local area are traversed according to the spatial priority from left to right and from top to bottom. After all target set points in the local area are traversed and measured, enter the next local significant area and return to step S4 to select a target set point for distance measurement.

8. The method for measuring the inclination of an ultra-high voltage transmission line tower according to claim 1, characterized in that: The step S7 uses N sets of target set point pair depth information and the image rigid transformation matrix to obtain the target set point pair depth information in the panoramic image of the power tower and calculate the inclination of the power tower. The specific steps are as follows: Step S7.1 Depth information mapping of power tower panoramic image According to the rigid transformation matrix T obtained in step S5, the distance d obtained in step S6 is mapped to the power tower panoramic image coordinate system based on the two-dimensional coordinates and ranging results of the effective measurement target point under the laser ranging sensor in step S6; combined with the high-definition camera external parameter matrix [R|t] in step S1.1.2, the three-dimensional coordinates (X g ,Y g ,Z g ), generate a three-dimensional point cloud set of target points as the basic data for inclination calculation; Step S7.2 3D coordinate point processing and tower centerline vector calculation Based on the 3D point cloud set obtained in step S7.1, the tower inclination is calculated by removing outliers, screening confidence levels, and performing plane fitting, as follows: The RANSAC algorithm is used to remove outliers; a plane is fitted to the 3D point cloud set in the local area; three point fitting planes are randomly selected as candidate planes, and the residuals from all points in the 3D point cloud set in the current local area to the candidate planes are calculated; Candidate plane equation aX+bY+cZ+d=0, residual Set the residual threshold, and the points exceeding the threshold are considered as outliers and eliminated, and those meeting the threshold conditions are judged as inliers; repeat the sampling N times, and select the candidate plane with the largest number of inliers as the final fitting result; For each local area, a candidate plane and its corresponding set of interior points are finally obtained for subsequent calculations; For each candidate plane, calculate the coordinates of its center point. Take any two candidate planes and calculate their geometric center coordinates O(x c ,y c ,z c ), O′(x c ′,y c ′,z c ′), connect the center points to get the center line vector Step S7.3 Calculation of tower inclination According to the center line vector of each candidate plane obtained and the plumb line vector Calculate the angle α between two vectors using the following formula: To convert radians to degrees, the formula is: That is the tower inclination angle θ. The calculated tower inclination angle can be divided into the degree of inclination in each dimension of the tower top, tower body and tower bottom, and then the inclination of the tower can be judged.

9. An ultra-high voltage transmission line tower inclination measuring device, characterized in that: include: Depth camera, including a high-definition camera and a laser ranging sensor eyepiece. The high-definition camera is used to obtain panoramic images of power poles and towers, and the laser ranging sensor eyepiece is used to obtain local images of power poles and towers, and complete initialization and preprocessing operations; memory for storing computer programs; A processor is configured to perform tower inclination measurement according to the method for measuring the inclination of an ultra-high voltage transmission line tower according to any one of claims 1 to 8 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, it can realize tower inclination measurement based on the method for measuring the inclination of an ultra-high voltage transmission line tower according to any one of claims 1 to 8.

Images