Power transmission tower pole deformation real-time monitoring system based on visual inspection

Through visual inspection and three-dimensional reconstruction technology, real-time acquisition and analysis of transmission tower deformation data and generation of three-dimensional models, solving the problems of incomplete data acquisition and environmental impact in the existing technology, real-time and accurate monitoring of transmission tower deformation is achieved, and improving the improvement of transmission tower deformation. The safety of the power system.

CN118936351APending Publication Date: 2024-11-12JINZHONG POWER SUPPLY COMPANY OF STATE GRID SHANXI ELECTRIC POWER
View PDF 0 Cites 6 Cited by

Patent Information

Application Number
CN202411212577.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-30
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

The existing transmission tower deformation monitoring technology is not comprehensive enough for data acquisition and is easily affected by the field environment, resulting in insufficient real-time and accurate monitoring.

Method used

Adopt a real-time monitoring system for deformation of the transmission tower based on visual detection, including the calibration module of the acquisition device, image acquisition module, stereo matching module, three-dimensional reconstruction module and deformation analysis module. Through image processing and three-dimensional reconstruction technology, real-time acquisition, matching and analysis towers are collected, matched and analyzed in real time. Rod deformation data generates three-dimensional models to realize real-time deformation monitoring.

Benefits of technology

Real-time and accurate monitoring of the deformation of transmission tower rods is achieved, monitoring accuracy and efficiency are improved, and the safety and reliability of the power system are enhanced.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118936351A_ABST
    Figure CN118936351A_ABST
Patent Text Reader

Abstract

The invention provides a power transmission tower pole deformation real-time monitoring system based on visual inspection, and relates to the technical field of power transmission and transformation. The image acquisition module acquires a plurality of real-time image information sets through a plurality of calibration acquisition devices; the stereo matching module extracts the characteristics of the plurality of real-time image information sets, carries out stereo matching on the plurality of real-time image information sets according to the plurality of image characteristics, and determines a stereo matching result; the three-dimensional reconstruction module performs three-dimensional reconstruction according to the stereo matching result to obtain a plurality of reconstruction results; a deformation analysis module performs deformation analysis on the plurality of reconstruction results according to an image acquisition time sequence to generate deformation judgment data; and the deformation monitoring alarm module carries out real-time deformation monitoring and generates an early warning instruction to carry out deformation alarm. The technical problem of insufficient real-time performance and accuracy of deformation monitoring of the power transmission tower pole in the prior art is solved, and the technical effect of accurately monitoring the deformation condition of the power transmission tower pole in real time is achieved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of power transmission and transformation technology, and in particular to a real-time monitoring system for power transmission tower deformation based on visual detection. Background Art

[0002] In the field of power transmission, transmission towers are key infrastructure that bear the important task of supporting high-voltage transmission lines. However, due to long-term exposure to the natural environment, affected by wind loads, temperature changes, material aging and other factors, transmission towers may deform. If this deformation is not discovered and handled in time, it may cause the tower to collapse and cause serious power accidents.

[0003] Existing deformation monitoring methods mainly include manual inspection, monitoring based on fixed sensors and some visual monitoring technologies. Manual inspection is the most traditional method, which detects tower deformation through regular inspections, but this method is inefficient, has a long cycle, and is difficult to monitor deformation in real time. Monitoring based on fixed sensors, such as mechanical sensors and laser scanners, can collect data in real time, but is limited by the installation location, has a limited monitoring range, and the reliability of sensors in bad weather is questionable. Some visual monitoring technologies, such as single camera monitoring, can capture changes in the appearance of towers, but lack three-dimensional information, which limits the accurate judgment of deformation. Summary of the invention

[0004] The present application provides a real-time monitoring system for transmission tower deformation based on visual detection, which solves the technical problems that the existing deformation monitoring technology is not comprehensive enough in collecting deformation data of transmission towers and is easily affected by the field environment, resulting in insufficient real-time and accuracy of monitoring. It achieves the technical effect of real-time and accurate monitoring of the deformation of transmission towers, thereby ensuring the safety and stability of power transmission.

[0005] In view of the above problems, the present application provides a real-time monitoring system for the deformation of a transmission tower based on visual detection, the system comprising: a collection device calibration module, the collection device calibration module is used to traverse the target area to calibrate multiple image collection devices based on the positioning point of the transmission tower, and determine multiple calibrated collection devices; an image collection module, the image collection module is used to collect images of the transmission tower in real time through the multiple calibrated collection devices, and obtain multiple real-time image information sets of the transmission tower; a stereo matching module, the stereo matching module is used to extract features based on the multiple real-time image information sets, generate multiple image features, and match the image features according to the image information sets. The plurality of image features are used to perform stereo matching on the plurality of real-time image information sets to determine a stereo matching result; a three-dimensional reconstruction module is used to construct the three-dimensional space coordinates of the transmission tower pole according to the stereo matching result to perform three-dimensional reconstruction to obtain a plurality of reconstruction results; a deformation analysis module is used to perform deformation analysis on the plurality of reconstruction results according to the image acquisition sequence to generate deformation determination data; a deformation monitoring and alarm module is used to perform real-time deformation monitoring on the transmission tower pole based on the deformation determination data, and generate an early warning instruction to be sent to a remote monitoring terminal to issue a deformation alarm on the transmission tower pole.

[0006] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0007] The acquisition device calibration module traverses the target area to calibrate multiple image acquisition devices based on the positioning points of the transmission tower pole, determines multiple calibration acquisition devices, ensures that each device can accurately locate the key parts of the transmission tower pole, and improves the accuracy of image acquisition. The image acquisition module acquires images of the transmission tower pole in real time through the multiple calibration acquisition devices, obtains multiple real-time image information sets of the transmission tower pole, and provides a basic data source for subsequent processing and analysis. The stereo matching module extracts features based on the multiple real-time image information sets, generates multiple image features, and performs stereo matching on the multiple real-time image information sets according to the multiple image features to determine the stereo matching results. Stereo matching is an important prerequisite for three-dimensional reconstruction, which ensures that images from different perspectives can be associated and matched. The three-dimensional reconstruction module constructs the three-dimensional spatial coordinates of the transmission tower pole according to the stereo matching results for three-dimensional reconstruction. This process converts the two-dimensional image information into a three-dimensional model, which can fully reflect the structure and deformation of the transmission tower pole. Through three-dimensional reconstruction, multiple reconstruction results are obtained, which reflect the three-dimensional state of the transmission tower pole at different time points, providing an intuitive and comprehensive perspective for deformation analysis. The deformation analysis module performs deformation analysis on the multiple reconstruction results according to the image acquisition sequence, analyzes the deformation amount and trend by comparing the three-dimensional models at different time points, and generates deformation determination data, which provides a scientific basis for deformation monitoring. The deformation monitoring and alarm module performs real-time deformation monitoring of the transmission tower based on the deformation determination data, generates early warning instructions and sends them to the remote monitoring terminal to issue deformation alarms for the transmission tower, ensuring timely response and processing when abnormalities are found.

[0008] In summary, this application forms a complete monitoring process from data collection, feature extraction, stereo matching, 3D reconstruction to deformation analysis through machine vision and 3D reconstruction technology. The system can not only monitor the deformation of transmission towers in real time, but also provide more accurate deformation judgment through learning and analysis, realize real-time monitoring of transmission tower deformation, improve the accuracy and efficiency of transmission tower deformation monitoring and environmental adaptability, provide effective technical support for the safe operation of the power system, and thus improve the safety and reliability of the power system.

[0009] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 A schematic diagram of the structure of a real-time monitoring system for transmission tower deformation based on visual detection provided in an embodiment of the present application;

[0011] Figure 2A schematic diagram of a process for determining a stereo matching result in a real-time monitoring system for transmission tower deformation based on visual detection provided in an embodiment of the present application;

[0012] Figure 3 A schematic diagram of the process of generating deformation determination data in a real-time monitoring system for transmission tower deformation based on visual detection provided in an embodiment of the present application.

[0013] Explanation of the reference numerals: acquisition device calibration module 10 , image acquisition module 20 , stereo matching module 30 , three-dimensional reconstruction module 40 , deformation analysis module 50 , deformation monitoring and alarm module 60 . DETAILED DESCRIPTION

[0014] The embodiment of the present application provides a real-time monitoring system for transmission tower deformation based on visual detection, and utilizes image processing technology and three-dimensional reconstruction to solve the technical problems that the existing deformation monitoring technology is not comprehensive enough in collecting deformation data of transmission towers, and is easily affected by the field environment, resulting in insufficient real-time and accuracy of monitoring. The system achieves the technical effect of real-time and accurate monitoring of the deformation of transmission towers, thereby ensuring the safety and stability of power transmission.

[0015] like Figure 1 As shown, the embodiment of the present application provides a real-time monitoring system for transmission tower deformation based on visual detection, and the system includes:

[0016] The acquisition device calibration module 10 is used to traverse the target area to calibrate multiple image acquisition devices based on the positioning points of the transmission tower pole, and determine multiple calibrated acquisition devices.

[0017] Specifically, the positioning points of transmission towers refer to points or areas on transmission towers that have significant features and are easy to identify, such as the top, corners or specific marking points of the towers. These points are the basis for image acquisition and processing and are used for calibration and positioning. The target area refers to the entire transmission tower area that needs to be monitored. Multiple image acquisition devices are cameras or sensors installed in different locations. They work together to capture tower images from multiple angles to provide a more comprehensive perspective.

[0018] First, select several representative and stable positioning points on the transmission tower, such as the top, bottom, corner or pre-marked feature points of the tower. These points have obvious visual distinction and are easy to be recognized by the image processing algorithm. Next, the acquisition device calibration module 10 uses the calibration tool to traverse the target area and calibrate multiple image acquisition devices one by one. Taking the checkerboard calibration method in the OpenCV library as an example, a checkerboard pattern with known dimensions is placed near the tower, and the checkerboard image is captured by the camera. According to the position of the corner points in the checkerboard, the camera's internal parameters (such as focal length, principal point coordinates) and distortion coefficient, as well as external parameters (such as position and direction) are calculated. This calibration process ensures that each camera can accurately align with the positioning point of the tower, eliminate or minimize lens distortion, and improve image accuracy.

[0019] The acquisition device calibration module 10 accurately calibrates the image acquisition device to ensure that the device can accurately capture and locate the key feature points of the transmission tower pole, thereby providing high-quality input data for subsequent image processing.

[0020] The image acquisition module 20 is used to acquire images of the transmission tower in real time through the multiple calibration acquisition devices to obtain multiple real-time image information sets of the transmission tower.

[0021] Specifically, the real-time image information set is a set of transmission tower image data collected continuously and in real time by multiple calibration acquisition devices. These images contain detailed visual information of the transmission tower at a specific time point and are used to monitor the tower status.

[0022] The image acquisition module 20 uses multiple calibrated image acquisition devices to continuously acquire images of the transmission tower pole in real time, providing a reliable data source for deformation monitoring. The calibrated image acquisition devices are fixed at multiple positions around the transmission tower pole, including the front, side and rear of the tower pole, to obtain images from different perspectives. In order to ensure image quality, the camera must also be equipped with an automatic adjustment function that can adapt to various light conditions. The image acquisition module 20 uses a control algorithm to trigger multiple image acquisition devices in real time to capture images and obtain multiple real-time image information sets of the transmission tower pole. For example, the camera can be set to shoot at a fixed frequency (such as once per second or once per minute), or the capture frequency can be automatically adjusted according to light changes.

[0023] The stereo matching module 30 is used to perform feature extraction based on the multiple real-time image information sets to generate multiple image features, perform stereo matching on the multiple real-time image information sets according to the multiple image features, and determine a stereo matching result.

[0024] Specifically, image features are representative information extracted from images, such as edges, corners, textures, etc. Stereo matching compares the same feature points in images from different perspectives and determines their relative positions in three-dimensional space to build a stereo model. The stereo matching result contains the three-dimensional coordinate information of the matching feature points, which is used for three-dimensional reconstruction and deformation analysis.

[0025] First, the module extracts features from multiple real-time image information sets. This process uses feature detection algorithms such as SIFT (Scale Invariant Feature Transform), SURF (Speeded Up Robust Features), or ORB (Oriented FAST and RotatedBRIEF) to identify key points in the image. For example, the SIFT (Scale Invariant Feature Transform) algorithm detects image gradients of different scales and directions to find unique feature points that can be identified even when the lighting changes or the perspective is rotated. Next, the module performs feature matching by comparing the feature points extracted from different images to find the correspondence between them. Finally, the module performs stereo matching based on the matched feature points, calculates the relative position of the feature points in three-dimensional space, and generates stereo matching results. This process can use triangulation or structure from motion (SfM) technology to calculate the three-dimensional coordinates of the matching points based on known camera intrinsic and extrinsic parameters.

[0026] The stereo matching module 30 extracts and matches feature points from multi-angle images, ensuring that the monitoring system can obtain three-dimensional spatial information from two-dimensional images and provide accurate stereo matching results for three-dimensional reconstruction.

[0027] The three-dimensional reconstruction module 40 is used to construct the three-dimensional space coordinates of the transmission tower according to the stereo matching result to perform three-dimensional reconstruction and obtain multiple reconstruction results.

[0028] Specifically, the 3D reconstruction module 40 realizes 3D reconstruction of the transmission tower pole by constructing the 3D spatial coordinates of the transmission tower pole based on the stereo matching results, generates multiple reconstruction results, and converts the multi-view image information into a 3D model. The multiple reconstruction results refer to 3D models of multiple transmission tower poles generated based on the stereo matching results at different time points or different viewpoints.

[0029] The module first receives the matching feature point information from the stereo matching module 30, which contains the relative position of each feature point in the three-dimensional space. The module uses this information, combined with the camera's internal and external parameters, such as focal length, position, and direction angle, to calculate the precise coordinates of the feature points in the three-dimensional space through triangulation or structure from motion (SfM) algorithm. For example, the module can calculate the XYZ coordinates of the matching points in the three-dimensional space by the known camera position and the pixel coordinates of the matching points in different images.

[0030] Next, the module organizes these 3D coordinate points to construct a 3D model of the transmission tower. This process can use point cloud processing technology to treat all feature points as point cloud data, and generate a continuous 3D mesh model through operations such as point cloud aggregation, smoothing, and meshing. This 3D reconstruction process can be repeated to generate multiple different reconstruction results based on the stereo matching results at different time points, which can be used to track the changes of the transmission tower over time.

[0031] Through the steps, the three-dimensional reconstruction module 40 can construct a three-dimensional model of the transmission tower from the multi-view images, providing intuitive and detailed data support for the deformation analysis and status monitoring of the monitoring system.

[0032] The deformation analysis module 50 is used to perform deformation analysis on the multiple reconstruction results according to the image acquisition sequence to generate deformation determination data.

[0033] Specifically, the image acquisition timing refers to the time sequence of image acquisition, ensuring that the deformation analysis is based on data at consecutive time points. The deformation determination data is generated based on the deformation analysis and is used to determine whether the transmission tower pole has deformed and the degree of deformation.

[0034] First, the reconstruction results of multiple time points are sorted and prepared according to the timing of image acquisition to ensure the continuity and accuracy of the analysis. Then, based on these models, the module uses point cloud comparison, mesh deformation analysis and other techniques to quantify the changes between models. For example, the ICP (Iterative Closest Point) algorithm is used for point cloud registration, the model point clouds at different time points are compared, and the displacement vector between points is calculated to quantify the degree of deformation. Then, the module further analyzes the deformation trend and detects the deformation pattern by statistical methods. For example, PCA (principal component analysis) is applied to identify and quantify the deformation trend of transmission tower poles in the main directions. In addition, the module may also use deep learning techniques such as CNN (convolutional neural network) or RNN (recurrent neural network) to identify and predict deformation patterns. These models can learn the laws of deformation based on historical data and predict future deformation trends. Finally, based on the results of deformation analysis, deformation judgment data is generated.

[0035] The deformation analysis module 50 detects and quantifies the deformation of the transmission tower by analyzing the changes of the three-dimensional model over time, thereby ensuring the accuracy and reliability of the monitoring results.

[0036] The deformation monitoring and alarm module 60 is used to perform real-time deformation monitoring of the transmission tower pole based on the deformation determination data, generate an early warning instruction and send it to the remote monitoring terminal to issue a deformation alarm for the transmission tower pole.

[0037] Specifically, the warning instruction is an instruction generated by the deformation monitoring alarm module 60 when abnormal deformation is detected, which is used to trigger an alarm of the remote monitoring terminal. The remote monitoring terminal is a remote monitoring center or mobile device that receives alarm information and is used to monitor the status of the transmission tower in real time.

[0038] The deformation monitoring and alarm module 60 receives deformation determination data in real time, and monitors and determines the deformation of the transmission tower pole through the set safety threshold and algorithm. Once abnormal deformation is detected, the module immediately generates an early warning instruction, and sends the early warning instruction to the remote monitoring terminal through the integrated communication system to ensure timely deformation alarm and take corresponding measures. Early warning notifications can use communication methods such as text messages, phone calls, emails or dedicated mobile application notifications to ensure that the alarm information can be conveyed to the monitoring personnel in a timely and accurate manner, thereby improving the real-time and safety of the transmission tower pole monitoring system, and providing important guarantees for timely detection and handling of abnormal deformation of transmission tower poles.

[0039] Furthermore, the stereo matching module 30 in the embodiment of the present application is also used to perform the following steps:

[0040] Grayscale processing is performed on the multiple real-time image information sets, and noise is removed according to the grayscale processing results to generate multiple real-time denoised image information sets; the multiple real-time denoised image information sets are segmented according to preset distances to generate multiple image segments; a convolutional neural network is used to traverse the multiple image segments to perform feature selection to obtain multiple segment features; the multiple segment features are integrated to determine the multiple image features.

[0041] Specifically, the real-time denoised image information set refers to a real-time image data set after noise removal. The preset distance is a fixed or adjustable size for image segmentation, which is used to control the size of the segmented image fragment. The image fragment is a sub-image obtained by segmenting the original image according to the preset distance. The fragment feature is a distinguishing feature extracted from the image fragment by a convolutional neural network.

[0042] Before performing feature extraction, the stereo matching module 30 performs a series of preprocessing on multiple real-time image information sets, including grayscale processing, noise removal, image segmentation and feature selection, to ensure the accuracy and efficiency of feature extraction. First, the module converts the color image into a grayscale image through grayscale processing. This process is achieved by converting RGB values ​​into grayscale values, which simplifies the image information and reduces the amount of calculation for subsequent processing. For example, the cv2.cvtColor() function in the OpenCV library is used to convert the image from the RGB color space to the grayscale space. Then, the grayscale image is denoised. Commonly used noise removal methods include median filtering, Gaussian filtering or bilateral filtering. For example, the median filter uses a sliding window to take the median of the pixels in the window as the new value of the center pixel, effectively removing salt and pepper noise.

[0043] After removing the noise, the module segments the denoised image according to the preset distance to generate multiple image segments. The preset distance can be set based on the tower features or camera parameters to ensure that key features are fully covered. For example, setting the segmentation size to 1 / 5 of the image width ensures that each part of the tower is analyzed separately. When performing fine extraction and segmentation, the preset distance can be set smaller to include more feature points; when performing coarse extraction, the segmentation distance can be set larger to include fewer feature points; it can also be segmented unequally and local fine extraction can be performed as required.

[0044] Then, the module uses convolutional neural networks (CNNs) to perform feature selection on the image segments. Convolutional neural networks (CNNs) use convolutional layers to identify local features in the image, pooling layers to reduce spatial dimensions, and fully connected layers to combine features. For example, pre-trained models such as VGG16 or ResNet50 are used to extract features from image segments. The feature selection process also includes: extracting color features using color histograms, color moments, etc.; extracting texture features using gray-level co-occurrence matrices (GLCMs), local binary patterns (LBPs), Gabor texture features, etc.; extracting shape features using edge detection (such as Canny, Sobel operators), Hough transforms, contour extraction, etc.

[0045] Finally, the module integrates multiple fragment features to determine the final image features. This process can use feature fusion or feature aggregation algorithms, such as feature weighted averaging or feature splicing, to ensure the comprehensiveness and accuracy of image features.

[0046] For example, the transmission tower images taken from four different angles are divided into 16 image segments after grayscale and noise removal. Feature extraction is performed on each image segment, including: calculating the histogram of the RGB color space to analyze the color distribution of the tower; calculating the color moment to describe the central tendency and distribution characteristics of the color. The texture contrast, energy and entropy of the tower surface are calculated using the gray level co-occurrence matrix (GLCM); the local texture pattern of the tower is extracted using the local binary pattern (LBP); and the specific texture direction of the tower surface is extracted using the Gabor filter. The tower edge is detected using the Canny operator; the straight part of the tower is detected using the Hough transform to locate the tower structure; and the complete outline information of the tower is obtained through contour extraction. The deep features of the tower image are automatically learned through the pre-trained ResNet model to capture the complex structure and details of the tower. 16 groups of segment features are obtained through feature extraction. Finally, these 16 groups of features are integrated through the feature splicing algorithm to form a complete image feature for the subsequent stereo matching process.

[0047] Further, such as Figure 2 As shown, the stereo matching module 30 in the embodiment of the present application is also used to perform the following steps:

[0048] Perform corner detection and matching based on the multiple image features and the multiple real-time image information sets to determine multiple matching corner point data; perform edge detection and matching based on the multiple image features and the multiple real-time image information sets to determine multiple matching edge data; perform disparity calculation based on the multiple matching corner point data and the multiple matching edge data to generate a disparity map; perform global optimization on the disparity map, perform quantitative evaluation based on the disparity optimization map, and determine the stereo matching result.

[0049] Specifically, matching corner points refers to corresponding corner points identified in images taken from different perspectives. By matching corner points, the corresponding relationship between images from different perspectives can be effectively determined. Matching edges refers to corresponding edges identified in images taken from different perspectives. By matching edges, the corresponding relationship between images can be further confirmed, especially when corner points are difficult to identify or limited in number, edge matching can provide additional information to assist in three-dimensional reconstruction. After acquiring multiple image features, the stereo matching module 30 determines multiple matching corner point data and matching edge data through corner point detection matching and edge detection matching, and then calculates disparity, generates a disparity map, and finally globally optimizes and quantitatively evaluates the disparity map to determine the stereo matching result. This process utilizes a variety of image processing and machine vision technologies to ensure the accuracy and reliability of stereo matching.

[0050] First, the module uses a corner detection algorithm, such as Harris or Shi-Tomasi, to detect corners from multiple image features, and then searches for matching points of these corners in the real-time image information set. In the image, corners are feature points with unique directionality and stability. For example, using feature matching algorithms such as SIFT or ORB, corresponding point pairs are found based on the similarity of feature descriptors. Next, edge detection algorithms such as Canny or Sobel operators are used to detect image edges, and matches of these edges are searched in the real-time image information set. Among them, edges are areas in the image where the grayscale value changes sharply, reflecting the contours of objects. For example, Hough transform is used to detect straight lines or circles, and the same edge features in different images are found.

[0051] Then, based on the matched corner and edge information, the disparity of corresponding points in different views, i.e., the pixel offset in the horizontal direction, is calculated through triangulation or cost aggregation-based algorithms to generate a disparity map. Among them, the disparity map is an image representing disparity information, which is used to show the difference between images from different perspectives. For example, using BlockMatching or Semi-Global BlockMatching algorithms, the best matching point is found for each pixel to calculate the disparity. The generated disparity map is globally optimized, such as using algorithms such as Graph Cuts, BeliefPropagation or MRF to eliminate noise and improve the accuracy of disparity estimation, and obtain an optimized disparity map, i.e., a disparity optimization map. The quality of the optimized disparity map is evaluated by calculating indicators such as the mean square error and structural similarity index of the disparity optimization map to ensure the reliability of the stereo matching results. Based on the disparity optimization map, the stereo matching results are determined, including matching point pairs, disparity information, and three-dimensional point cloud data.

[0052] Furthermore, the 3D reconstruction module 40 in the embodiment of the present application is also used to perform the following steps:

[0053] Extract the built-in parameters of the multiple calibration acquisition devices, and calculate the baseline distance of the multiple calibration acquisition devices; convert the disparity optimization map into a depth map according to the built-in parameters and the baseline distance; extract the external parameters of the multiple calibration acquisition devices, the external parameters include the rotation matrix and translation vector of the multiple calibration acquisition devices; traverse the target area based on the multiple calibration acquisition devices to construct a spatial coordinate system of the target area; traverse each pixel point of the depth map based on the rotation matrix and the translation vector and convert it to the spatial coordinate system to determine the three-dimensional point cloud data; perform point cloud segmentation based on the three-dimensional point cloud data to determine the three-dimensional spatial coordinates of the transmission tower.

[0054] Specifically, built-in parameters are parameters that describe the internal properties of the camera, including focal length, principal point position, pixel size, etc., which are used to correct image distortion and establish the relationship between pixel coordinates and physical coordinates. The baseline distance is the distance between the center points of the two image acquisition devices, which is the basis for calculating parallax and depth. The depth map is an image that represents the distance from each pixel to the camera, and is an important intermediate result of three-dimensional reconstruction in stereo vision. External parameters are parameters that describe the spatial position and posture of the camera, including rotation matrix and translation vector, which are used to convert pixel points from image coordinates to world coordinates. Among them, the rotation matrix describes the rotation of an object in three-dimensional space; the translation vector describes the translation of an object in three-dimensional space.

[0055] The three-dimensional reconstruction module 40 finally determines the three-dimensional spatial coordinates of the transmission tower pole through the steps of depth map conversion, point cloud generation, point cloud segmentation, etc. based on the disparity map, built-in parameters, external parameters and baseline distance. First, the built-in parameters such as focal length and principal point position are read from the camera parameter file. The baseline distance is calculated using the relative position of the two calibration acquisition devices. The disparity optimization map is converted into a depth map using the built-in parameters and the baseline distance. Each pixel value of the depth map represents the absolute distance from the corresponding point in the scene to the camera. According to the traversal path of the calibration acquisition device, a spatial coordinate system covering the target area is constructed. This coordinate system provides a reference frame for subsequent point cloud conversion and segmentation. The rotation matrix and translation vector of the calibration acquisition device are read, and these parameters describe the position and posture of the camera in the spatial coordinate system. Based on the rotation matrix, translation vector and depth map, each pixel point is converted from the image coordinate to the world coordinate to generate three-dimensional point cloud data. This process utilizes the principle of triangulation. The point cloud data of the transmission tower pole is separated from the three-dimensional point cloud using a point cloud segmentation algorithm, such as clustering based on Euclidean distance, region growth or plane fitting based methods. The segmented transmission tower point cloud data is analyzed to extract its three-dimensional coordinates in the spatial coordinate system, determine the three-dimensional position of the transmission tower, and provide accurate three-dimensional position information for subsequent deformation monitoring and maintenance work.

[0056] Furthermore, the 3D reconstruction module 40 in the embodiment of the present application is also used to perform the following steps:

[0057] Feature points are extracted based on the three-dimensional spatial coordinates of the transmission tower pole to obtain multiple three-dimensional feature point coordinates; the stereo matching results are sorted based on the image acquisition timing to generate a stereo matching timing; the three-dimensional spatial coordinates of the transmission tower pole are three-dimensionally reconstructed in sequence according to the stereo matching timing in combination with the multiple three-dimensional feature point coordinates to obtain the multiple reconstruction results.

[0058] Specifically, the three-dimensional feature point coordinates are the feature point coordinates extracted from the three-dimensional space coordinates. Stereo matching timing is to sort the stereo matching results according to the image acquisition timing, which is used to track and analyze the changes of three-dimensional data over time.

[0059] From the three-dimensional spatial coordinates of the transmission tower obtained in the previous step, the feature point detection algorithm is used to extract the coordinates of multiple three-dimensional feature points. These feature points usually have unique textures or structures, such as corners, connection points or special marks of the tower, which are convenient for subsequent matching and analysis. According to the image acquisition time sequence, the previous stereo matching results are sorted to generate a stereo matching time sequence. This sorting ensures the temporal continuity of the data and facilitates the analysis of the deformation of the transmission tower over time. Combining the stereo matching time sequence and the three-dimensional feature point coordinates, the three-dimensional spatial coordinates of the transmission tower are reconstructed in sequence. This process can use point cloud fusion, meshing or surface reconstruction algorithms to generate a three-dimensional model of the transmission tower at different time points. The reconstruction results include not only the shape and position information of the tower, but also may contain details such as texture and color to form a complete three-dimensional reconstruction result. These reconstruction results can not only be used to intuitively display the three-dimensional structure and deformation of the transmission tower, but also can be used to accurately analyze the deformation trend of the tower over time, so as to achieve accurate monitoring of the changes of the transmission tower over time.

[0060] Further, such as Figure 3 As shown, the deformation analysis module 50 in the embodiment of the present application is also used to perform the following steps:

[0061] The multiple reconstruction results are three-dimensionally aligned according to the image acquisition timing, and the multiple reconstruction results are divided into adjacent groups according to the alignment results to generate a three-dimensional reconstruction pair; the coordinate point difference of the three-dimensional reconstruction pair is calculated, and the deformation degree of the transmission tower is determined according to the difference calculation result; it is determined whether the deformation degree of the transmission tower is greater than or equal to a preset deformation threshold; if the deformation degree of the transmission tower is greater than or equal to the preset deformation threshold, it is regarded that the transmission tower is deformed, a deformation label is generated, and the deformation label is added to the deformation judgment data.

[0062] Specifically, a 3D reconstruction pair is two groups of reconstruction results divided according to time sequence proximity, which are used to calculate the degree of deformation. The coordinate point difference is the change in the position of the corresponding coordinate point in the 3D reconstruction pair, which is used to quantify the deformation. Deformation is an indicator that describes the degree of deformation of an object. The preset deformation threshold is a threshold used to determine whether the deformation is significant. If this threshold is exceeded, deformation is considered to exist. The deformation label is a reconstruction result that marks the presence of deformation, which is convenient for subsequent analysis and processing.

[0063] The deformation analysis module 50 detects the deformation of the transmission tower pole by sequential three-dimensional alignment, deformation calculation, threshold judgment and deformation label generation. According to the image acquisition sequence, multiple reconstruction results are three-dimensionally aligned to ensure that the models at different time points accurately correspond in space. Subsequently, the aligned reconstruction results are divided into adjacent groups to generate three-dimensional reconstruction pairs. For example, the reconstruction results of the first day and the second day are taken as one pair, the reconstruction results of the second day and the third day are taken as another pair, and so on.

[0064] Calculate the position difference of the corresponding coordinate points in each pair of 3D reconstructions. Based on these differences, use statistical methods, such as the root mean square error (RMSE), to calculate the coordinate difference of the key points of the tower between two time points, quantify the deformation degree of the transmission tower, and obtain the deformation degree. Set the threshold for deformation judgment according to the application scenario and actual needs. These thresholds can be absolute values, such as the maximum point displacement does not exceed a certain value; or relative values, such as the deformation rate does not exceed a certain percentage. Compare the calculated deformation degree with the preset deformation threshold. If the deformation degree is greater than or equal to the preset threshold, it is considered that the transmission tower is deformed, and a deformation label is generated for the reconstruction result in the deformation judgment data to mark it as deformed. If the deformation degree of the transmission tower is less than the preset deformation threshold, it is considered that the transmission tower is not deformed.

[0065] Exemplarily, within a week, the transmission tower pole is reconstructed every day, and a total of 7 reconstruction results are obtained. The deformation analysis module 50 first performs three-dimensional alignment on the 7 reconstruction results according to the image acquisition sequence to ensure their accurate correspondence in space. Subsequently, the 7 reconstruction results are divided into adjacent groups to form 6 pairs of 3D reconstruction pairs. For each pair of 3D reconstruction pairs, the module calculates the difference between the corresponding coordinate points. For example, in the reconstruction results of the 1st day and the 2nd day, the difference in the coordinate points of the key parts of the tower pole is an average of 2mm. Based on these differences, the root mean square error RMSE is used to calculate the deformation degree, and the obtained deformation degree is 2.5mm. The preset deformation threshold is 3mm. Since the deformation degree of 2.5mm is less than the preset threshold, the module does not generate a deformation label for the reconstruction pair. The deformation degree between the 5th day and the 6th day is calculated to be 3.5mm, which exceeds the preset threshold of 3mm. Therefore, the module generates a deformation label for the reconstruction pair and adds this deformation label to the deformation judgment data to mark the existence of deformation of the transmission tower pole during this time period.

[0066] Through the above steps, the deformation analysis module 50 can accurately detect and quantify the deformation of the transmission tower. By monitoring the deformation trend of the tower over time, it can warn of potential structural problems and take maintenance measures in time to ensure the safe operation of the power system.

[0067] Furthermore, the deformation monitoring alarm module 60 of the embodiment of the present application is also used to perform the following steps:

[0068] When the deformation determination data contains the deformation tag, the monitoring mechanism is activated; the transmission tower pole is continuously deformed by the monitoring mechanism to obtain multi-dimensional deformation data, wherein the multi-dimensional deformation data includes deformation angle data, deformation displacement data, and deformation stress data; the deformation displacement data is analyzed based on the deformation angle data combined with the deformation stress data to generate a deformation abnormality analysis result; an early warning condition is triggered based on the deformation abnormality analysis result, and the early warning instruction is generated.

[0069] Specifically, the monitoring mechanism is a deformation detection and analysis system that is activated when deformation occurs. Multidimensional deformation data is a set of deformation data that contains multiple information such as deformation angle, displacement and stress. Deformation angle data is data that describes the angle change of the transmission tower in space. Deformation displacement data is data that describes the position change of the transmission tower, and is an important parameter for deformation monitoring. Deformation stress data is data that describes the stress state of the transmission tower, and is used to analyze the cause of deformation. The deformation anomaly analysis result is the result of evaluating the deformation displacement by analyzing the deformation angle and stress data. Warning conditions refer to the conditions that trigger an alarm, such as deformation displacement exceeding a safety threshold.

[0070] When a deformation tag appears in the deformation determination data, the monitoring mechanism is automatically activated and starts to perform continuous deformation detection on the transmission tower. The deformation angle, displacement, stress and other data of the transmission tower are continuously detected to form multi-dimensional deformation data. For example, the angle change and stress condition are monitored in real time by sensors installed on the tower. Based on the deformation angle data and deformation stress data, the deformation displacement data is analyzed to evaluate the deformation anomaly. For example, when the deformation angle and stress data indicate that the tower is subjected to abnormal loads, the displacement data is analyzed to determine whether the deformation exceeds the safety range. According to the deformation anomaly analysis results, it is determined whether the warning condition is met. If the deformation displacement exceeds the preset safety threshold, the system triggers the warning condition and generates a warning instruction. For example, when the deformation displacement of the tower is detected to exceed 10mm, the system generates a warning instruction to indicate possible structural risks. Through the above steps, the deformation monitoring alarm module 60 can monitor the deformation of the transmission tower in real time, warn of potential risks in a timely manner, and ensure the safe operation of power facilities.

[0071] Furthermore, the deformation monitoring alarm module 60 of the embodiment of the present application is also used to perform the following steps:

[0072] Calculation is performed based on the deformation angle data, the deformation displacement data, and the deformation stress data to determine the deformation rate; multiple risk levels are constructed, and the transmission tower is marked based on the deformation rate combined with the multiple risk levels to determine the deformation risk level of the transmission tower; the three-dimensional space coordinates of the transmission tower are traversed to perform deformation positioning and determine multiple deformation points; and the warning conditions are constructed based on the multiple deformation points combined with the deformation risk level.

[0073] Specifically, the deformation rate describes the speed at which the deformation of the transmission tower changes over time and is an important indicator for measuring the deformation trend. The risk level is a level divided according to the severity of deformation and is used to evaluate and identify deformation risks. The deformation risk level is the deformation risk level of the transmission tower determined in combination with the deformation rate. The deformation point is the specific coordinate point on the transmission tower where deformation occurs.

[0074] Based on the deformation angle data, deformation displacement data and deformation stress data, the deformation rate of the transmission tower pole is calculated. For example, the deformation rate is obtained by calculating the ratio of the deformation displacement difference between two consecutive time points to the time interval. Multiple risk levels are defined, such as low risk, medium risk and high risk, and each level corresponds to a different deformation rate threshold. The deformation rate is compared with the risk level threshold to determine the deformation risk level of the transmission tower pole. For example, if the deformation rate is 0.2 mm per day, it may be marked as low risk; if the deformation rate is 1 mm per day, it may be marked as high risk. The three-dimensional spatial coordinates of the transmission tower pole are traversed to identify the specific location where the deformation occurs, and multiple deformation points are determined. For example, by comparing the three-dimensional reconstruction results at different time points, the coordinate point with the most significant deformation is identified. Based on the deformation point position and risk level, the warning condition is set. For example, when the deformation risk level of a specific deformation point position reaches a high risk, a warning is triggered.

[0075] The deformation monitoring and warning module 60 achieves accurate assessment and early warning of the deformation risk of the transmission tower by calculating the deformation rate, establishing the risk level, determining the deformation point and establishing the early warning conditions.

[0076] In summary, the real-time monitoring system for transmission tower deformation based on visual detection provided in the embodiments of the present application has the following technical effects:

[0077] The acquisition device calibration module 10 traverses the target area to calibrate multiple image acquisition devices based on the positioning points of the transmission tower pole, determines multiple calibration acquisition devices, ensures that each device can accurately locate the key parts of the transmission tower pole, and improves the accuracy of image acquisition. The image acquisition module 20 acquires images of the transmission tower pole in real time through the multiple calibration acquisition devices, obtains multiple real-time image information sets of the transmission tower pole, and provides a basic data source for subsequent processing and analysis. The stereo matching module 30 extracts features based on the multiple real-time image information sets, generates multiple image features, and performs stereo matching on the multiple real-time image information sets according to the multiple image features to determine the stereo matching results. Stereo matching is an important prerequisite for three-dimensional reconstruction, which ensures that images under different perspectives can be associated and matched. The three-dimensional reconstruction module 40 constructs the three-dimensional space coordinates of the transmission tower pole according to the stereo matching results for three-dimensional reconstruction. This process converts the two-dimensional image information into a three-dimensional model, which can fully reflect the structure and deformation of the transmission tower pole. Through three-dimensional reconstruction, multiple reconstruction results are obtained, which reflect the three-dimensional state of the transmission tower pole at different time points, providing an intuitive and comprehensive perspective for deformation analysis. The deformation analysis module 50 performs deformation analysis on the multiple reconstruction results according to the image acquisition sequence, analyzes the deformation amount and trend by comparing the three-dimensional models at different time points, and generates deformation determination data, which provides a scientific basis for deformation monitoring. The deformation monitoring and alarm module 60 performs real-time deformation monitoring of the transmission tower based on the deformation determination data, generates an early warning instruction and sends it to the remote monitoring terminal to issue a deformation alarm for the transmission tower, ensuring timely response and processing when abnormalities are found.

[0078] In general, the embodiment of the present application forms a complete monitoring process from data collection, feature extraction, stereo matching, 3D reconstruction to deformation analysis through machine vision and 3D reconstruction technology. The system can not only monitor the deformation of transmission towers in real time, but also provide more accurate deformation judgment through learning and analysis, realize real-time monitoring of transmission tower deformation, improve the accuracy and efficiency of transmission tower deformation monitoring and environmental adaptability, provide effective technical support for the safe operation of the power system, and thus improve the safety and reliability of the power system.

[0079] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A real-time monitoring system for transmission tower deformation based on visual detection, characterized in that: The system comprises: A collection device calibration module, the collection device calibration module is used to traverse the target area to calibrate multiple image collection devices based on the positioning points of the transmission tower pole, and determine multiple calibrated collection devices; An image acquisition module, the image acquisition module is used to acquire images of the transmission tower in real time through the multiple calibration acquisition devices to obtain multiple real-time image information sets of the transmission tower; A stereo matching module, the stereo matching module is used to perform feature extraction based on the multiple real-time image information sets to generate multiple image features, perform stereo matching on the multiple real-time image information sets according to the multiple image features, and determine a stereo matching result; A three-dimensional reconstruction module, wherein the three-dimensional reconstruction module is used to construct the three-dimensional space coordinates of the transmission tower pole according to the stereo matching result to perform three-dimensional reconstruction and obtain multiple reconstruction results; A deformation analysis module, the deformation analysis module is used to perform deformation analysis on the multiple reconstruction results according to the image acquisition time sequence to generate deformation determination data; The deformation monitoring and alarm module is used to perform real-time deformation monitoring of the transmission tower pole based on the deformation determination data, generate an early warning instruction and send it to the remote monitoring terminal to issue a deformation alarm for the transmission tower pole.

2. The real-time monitoring system for transmission tower deformation based on visual detection according to claim 1, characterized in that: The stereo matching module is also used to perform the following steps: Performing grayscale processing on the multiple real-time image information sets, and removing noise according to the grayscale processing results to generate multiple real-time denoised image information sets; Segmenting the plurality of real-time denoised image information sets according to a preset distance to generate a plurality of image segments; Using a convolutional neural network to traverse the multiple image segments to perform feature selection and obtain multiple segment features; The multiple segment features are integrated to determine the multiple image features.

3. The real-time monitoring system for transmission tower deformation based on visual detection according to claim 1 is characterized in that: The stereo matching module is also used to perform the following steps: Performing corner point detection and matching based on the multiple image features and the multiple real-time image information sets to determine multiple matching corner point data; Perform edge detection and matching based on the multiple image features and the multiple real-time image information sets to determine multiple matching edge data; Perform disparity calculation according to the plurality of matching corner point data and the plurality of matching edge data to generate a disparity map; The disparity map is globally optimized, and a quantitative evaluation is performed based on the disparity optimization map to determine the stereo matching result.

4. The real-time monitoring system for transmission tower deformation based on visual detection according to claim 3 is characterized in that: The three-dimensional reconstruction module is also used to perform the following steps: Extracting built-in parameters of the plurality of calibration acquisition devices, and calculating baseline distances of the plurality of calibration acquisition devices; Converting the disparity optimization map into a depth map according to the built-in parameters and the baseline distance; Extracting external parameters of the multiple calibration acquisition devices, wherein the external parameters include rotation matrices and translation vectors of the multiple calibration acquisition devices; Traversing the target area based on the multiple calibration acquisition devices, constructing a spatial coordinate system of the target area; Based on the rotation matrix and the translation vector, each pixel point of the depth map is traversed and converted into the spatial coordinate system to determine three-dimensional point cloud data; Point cloud segmentation is performed based on the three-dimensional point cloud data to determine the three-dimensional spatial coordinates of the transmission tower.

5. The real-time monitoring system for transmission tower deformation based on visual detection according to claim 4 is characterized in that: The three-dimensional reconstruction module is also used to perform the following steps: Extracting feature points based on the three-dimensional spatial coordinates of the transmission tower pole to obtain a plurality of three-dimensional feature point coordinates; Sorting the stereo matching results based on the image acquisition timing to generate a stereo matching timing; The three-dimensional spatial coordinates of the transmission tower pole are three-dimensionally reconstructed in sequence according to the stereo matching sequence and in combination with the multiple three-dimensional feature point coordinates to obtain the multiple reconstruction results.

6. The real-time monitoring system for transmission tower deformation based on visual detection according to claim 1, characterized in that: The deformation analysis module is also used to perform the following steps: Performing three-dimensional alignment on the multiple reconstruction results according to the image acquisition time sequence, and dividing the multiple reconstruction results into adjacent groups according to the alignment result to generate three-dimensional reconstruction pairs; Calculating the difference in coordinate points of the three-dimensional reconstruction pair, and determining the deformation degree of the transmission tower pole according to the difference calculation result; Determining whether the deformation degree of the transmission tower pole is greater than or equal to a preset deformation threshold; If the deformation degree of the transmission tower is greater than or equal to the preset deformation threshold, the transmission tower is deemed to be deformed, a deformation label is generated, and the deformation label is added to the deformation determination data.

7. The real-time monitoring system for transmission tower deformation based on visual detection according to claim 6 is characterized in that: The deformation monitoring alarm module is also used to perform the following steps: When the deformation determination data contains the deformation tag, activating a monitoring mechanism; The monitoring mechanism is used to continuously detect the deformation of the transmission tower pole to obtain multi-dimensional deformation data, wherein the multi-dimensional deformation data includes deformation angle data, deformation displacement data, and deformation stress data; Analyzing the deformation displacement data based on the deformation angle data in combination with the deformation stress data to generate a deformation anomaly analysis result; A warning condition is triggered based on the deformation abnormality analysis result, and the warning instruction is generated.

8. The real-time monitoring system for transmission tower deformation based on visual detection according to claim 7 is characterized in that: The deformation monitoring alarm module is also used to perform the following steps: Calculating based on the deformation angle data, the deformation displacement data, and the deformation stress data to determine the deformation rate; Constructing multiple risk levels, marking the transmission tower pole based on the deformation rate combined with the multiple risk levels, and determining the deformation risk level of the transmission tower pole; Traversing the three-dimensional spatial coordinates of the transmission tower pole to perform deformation positioning and determine a plurality of deformation points; The warning condition is constructed based on the multiple deformation points and the deformation risk level.

Citation Information

Cited By

  • Method for evaluating effect of same-view-angle image generation before and after damage

    CN119338813A

  • Real-time displacement monitoring method and system for tension test of horizontal acting force of ground wire

    CN120125637A

  • A real-time displacement monitoring method and system for ground wire horizontal force tension test

    CN120125637B

  • Deformation detection method and system based on constructional engineering

    CN120408165A

  • Telegraph pole inclination angle detection and early warning method and related equipment

    CN120726571A