A method of fitting power lines from surveillance imagery
By constructing a power line monitoring image dataset and combining a semantic segmentation network and a catenary morphological equation, the problem of three-dimensional coordinate recognition of power lines in monitoring images was solved, achieving low-cost and high-precision power line detection, which is suitable for the regular inspection of transmission lines.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- NANJING MEIJISEN INFORMATION TECH CO LTD
- Filing Date
- 2025-01-22
- Publication Date
- 2026-07-24
AI Technical Summary
Existing technologies struggle to accurately identify and locate the three-dimensional spatial coordinates of power lines in surveillance footage, and traditional detection methods are either costly or inefficient, failing to effectively meet the needs of regular inspections of power transmission lines.
A power line monitoring image dataset was constructed. The world coordinates of the power lines were fitted by the semantic segmentation network Unet and the region growing algorithm, combined with the catenary morphology equation. Data augmentation and interpolation smoothing were used to generate power line masks. The camera optical center ray equation and vertical plane equation were solved simultaneously to achieve high-precision positioning of the power lines.
It achieves low-cost, high-precision three-dimensional spatial coordinate fitting of power lines, improving the efficiency and accuracy of power line detection, reducing detection costs, and is suitable for regular inspection of transmission lines.
Smart Images
Figure CN122453933A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of the power industry, and in particular to a method for fitting power lines from surveillance images. Background Technology
[0002] In power transmission systems, transmission lines, as an indispensable and crucial component, play a vital role in ensuring a safe and stable power supply. With the rapid progress of China's energy industry, the scale of transmission lines has been continuously expanding to meet the ever-increasing electricity demand. However, transmission line equipment is prone to aging, damage, and corrosion, problems caused by prolonged exposure to the elements, which seriously threaten the safety and reliability of the power system. Regular inspections of the distribution network are necessary to detect defects early and effectively plan maintenance activities.
[0003] Traditional power line inspection methods typically include ground-based and aerial inspections. However, high-voltage power towers are mostly built in sparsely populated, inaccessible, and environmentally diverse mountainous areas. Furthermore, manual inspection of transmission lines is extremely difficult, and helicopter-borne inspections are expensive. Therefore, regularly monitoring power lines by fixing cameras on towers is crucial. Accurately identifying and locating the three-dimensional spatial coordinates of power lines is a critical issue in observing power line faults. Some researchers have chosen to use LiDAR technology to acquire high-precision three-dimensional point cloud data of the surrounding environment for power line extraction; however, research on directly obtaining the three-dimensional spatial location of power lines from surveillance images remains lacking. Compared to the former, the latter requires less expensive hardware and can achieve higher accuracy. Summary of the Invention
[0004] This invention discloses a method for fitting world coordinates of power lines from surveillance images, with the aim of proposing a lower-cost solution for power line safety detection: constructing a power line surveillance image dataset that can be used for semantic segmentation network learning.
[0005] In three-dimensional space, power lines are not straight lines, but rather have a catenary shape. Overhead lines are further divided into suspension points of equal and unequal height; this invention focuses on suspension points of equal height. The equation for the catenary shape curve satisfies:
[0006] in : This is called specific load, which is the load per unit length converted to the value per unit area. It is a calculable constant that is related to its own load and the environment. : Horizontal component of axial stress in electric field lines.
[0007] However, there is no precise mathematical expression to describe the catenary-shaped electric field line in surveillance images. A further improvement of this invention lies in manually sampling an electric field line template from a real image, performing interpolation smoothing on the sampled template to restore the curve characteristics of the suspended electric field line in the flat-panel image as much as possible, and then processing the template with diverse grayscale distributions to generate an electric field line mask.
[0008] In the background of surveillance images, the sky and buildings on the ground each occupy roughly half of the image width. This invention performs data augmentation operations on the power line mask, including random rotation of -5° to 5°, mirroring the mask, and random vertical and horizontal translations of 0 to 10% of the image's width and height, followed by overlaying with similar surveillance background images to form a dataset.
[0009] The synthesized power line monitoring image dataset was used to train the semantic segmentation network Unet.
[0010] The power line pixels segmented by the Unet network are grouped using a region growing algorithm.
[0011] Initialize an empty queue. All target pixels As a seed point, it is added to the queue once. middle:
[0012] in It is the collection of all pixels.
[0013] Retrieve the first pixel of the queue Join Group ,get Adjacent target pixels Calculate the Manhattan distance between the two. :
[0014] in yes coordinates yes The coordinates.
[0015] Define Manhattan distance threshold ,if With calculation Satisfy the following formula:
[0016] Then Add to queue As a new seed point, and added to the current group. The algorithm iterates through the data until all target pixels are grouped, and finally returns the result of the electric field line grouping. .right Perform noise filtering, that is:
[0017] Groups that meet the above conditions are retained as valid power line groups. .
[0018] The equations for the vertical planes containing each power line are constructed based on predefined engineering parameters.
[0019] One end of the power line suspension point Point of suspension The coordinates of the point in the world coordinate system are The tower rotation angle can be expressed as the coordinates around the base. Axis rotation Angle, whose corresponding rotation matrix is So, regarding the suspension point of the tower The coordinates in the world coordinate system are:
[0020] Define auxiliary calculation points Its coordinates in the world coordinate system are:
[0021] according to Three coordinates construct a vector The cross product of the two vectors yields the normal vector of the equation of the plane perpendicular to the electric field lines. The equation of the plane perpendicular to the electric field lines is obtained as follows:
[0022] Construct the ray equations for each pixel of each set of power lines and the camera optical center.
[0023] From the above Take one of the groups Further obtain one of the pixels Then its corresponding pixel coordinates are Therefore, based on the camera intrinsic parameter matrix, the image plane coordinates can be obtained as follows: Its corresponding three-dimensional spatial coordinates in the camera coordinate system can be expressed as: ,in This refers to the camera's focal length. Based on the camera's extrinsic parameters, the image plane coordinates in the camera coordinate system are determined. Convert to world coordinate system ), combined with the optical center position, i.e., the equipment position The direction vector of the obtainable optical center ray equation
[0024] Further expressing the optical center ray equation as t represents the distance measured along the ray from the starting point.
[0025] Finally, the world coordinates of the electric field lines are calculated by solving the simultaneous equations of the camera optical center ray constructed from each set of electric field lines and their corresponding vertical plane equations. Transform the coordinate system to the vertical plane using the formula The complete world coordinates of the electric field lines can be obtained by performing least squares fitting. Attached Figure Description
[0026] The invention can be readily understood by carefully reading the following detailed description in conjunction with the accompanying drawings, which include: Figure 1 This is a flowchart of the method of the present invention; Figure 2 A schematic diagram of a semantic segmentation dataset for power line surveillance images; Figure 3 This is a schematic diagram showing the construction and parameters of a simulated power tower; Figure 4 This is a schematic diagram of the vertical plane of the power line; Figure 5 A schematic diagram of the camera's optical center ray; Figure 6 Line graph showing the statistical error of experiments with different sags; Figure 7 A spatial visualization of the point cloud for fitting electric field lines. Detailed Implementation
[0027] The accompanying drawings are for illustrative purposes only and should not be construed as limiting the scope of this patent. The technical solution of the present invention will be further described below in conjunction with the accompanying drawings and embodiments.
[0028] like Figure 1 As shown, this embodiment proposes a method for fitting world coordinates of power lines from surveillance images, including the following process: Synthetic power line surveillance video semantic segmentation dataset.
[0029] The dataset was used to train the Unet network to segment power lines, and the pixels were grouped using a region growing algorithm.
[0030] Based on predefined engineering parameters, construct the equation of the vertical plane where each power line is located, and construct the ray equation of each pixel point of each group of power lines and the optical center of the camera.
[0031] Solve the equations simultaneously to obtain the world coordinates of the electric field lines. Transform the coordinate system to the vertical plane and use the least squares method to fit the equations of the catenary to obtain the complete world coordinates of the electric field lines.
[0032] Specifically, in three-dimensional space, power lines are not straight lines, but rather have a catenary shape. Overhead lines are further divided into suspension points of equal and unequal height; this invention focuses on suspension points of equal height. The equation for the catenary shape curve satisfies:
[0033] in : This is called specific load, which is the load per unit length converted to the value per unit area. It is a calculable constant that is related to its own load and the environment. : Horizontal component of axial stress in electric field lines.
[0034] However, there is no precise mathematical expression to describe the catenary-shaped electric field line in surveillance images. A further improvement of this invention lies in manually sampling an electric field line template from a real image, performing interpolation smoothing on the sampled template to restore the curve characteristics of the suspended electric field line in the flat-panel image as much as possible, and then processing the template with diverse grayscale distributions to generate an electric field line mask.
[0035] In the background of the surveillance video, the sky and buildings on the ground each roughly occupy half of the image width. This invention performs data augmentation operations on the power line mask, including random rotation of -5° to 5°, mirroring the mask, and random vertical and horizontal translations of 0-10% of the image's width and height. The resulting image is then overlaid with a similar surveillance background image to form a dataset. A detailed illustration is shown below. Figure 2 As shown.
[0036] Specifically, this dataset was used to train a 5-layer Unet semantic segmentation network with skip connections. The training and test sets were split in a 9:1 ratio. The training platform used a Google Colab T4 GPU, and the model was trained for 100 epochs. Given an original image as input, the model outputs a segmentation map showing the locations of the corresponding power line pixels in the image.
[0037] Specifically, a region growing algorithm is used to group the power line pixels segmented by the Unet network. The pixel value of the target power line is \left [ {128,0,0} \right ] The background pixel value is \left [ {0,0,0} \right ] Initialize an empty queue. All target pixels As a seed point, it is added to the queue once. middle:
[0038] in It is the collection of all pixels.
[0039] Retrieve the first pixel of the queue Join Group ,get Adjacent target pixels Calculate the Manhattan distance between the two. :
[0040] in yes coordinates yes The coordinates.
[0041] Define Manhattan distance threshold ,if With calculation Satisfy the following formula:
[0042] Then Add to queue As a new seed point, and added to the current group. The algorithm iterates through the data until all target pixels are grouped, and finally returns the result of the electric field line grouping. .right Perform noise filtering, that is:
[0043] Groups that meet the above conditions are retained as valid power line groups. .
[0044] Specifically, the equations for the vertical planes containing each power line are constructed based on predefined engineering parameters. The predefined engineering parameters are shown in the table below, and their schematic diagram is shown in the figure below. Figure 3 As shown.
[0045]
[0046] The equipment is installed on tower 1, facing the center of tower 2, with the base of tower 1 as the origin of the world coordinate system. For example... Figure 4 The diagram shows a power line with one end suspended at a point denoted as . Point, suspension point The coordinates of the point in this world coordinate system are ),correspond Figure 3 The distances of 8, 9, and 10 in the coordinate system, and the tower rotation angle can be expressed as coordinates around the base. Axis rotation Angle, such as Figure 3 As shown in Figure 7, its corresponding rotation matrix is So, regarding the suspension point of tower 2 The coordinates in the world coordinate system are:
[0047] in The difference in the position of the tower corresponds to Figure 3 The distances of 4, 5, and 6 in the text.
[0048] Define auxiliary calculation points Its coordinates in the world coordinate system are:
[0049] according to Three coordinates construct a vector The cross product of the two vectors yields the normal vector of the equation of the plane perpendicular to the electric field lines. The equation of the plane perpendicular to the electric field lines is obtained as follows:
[0050] Specifically, the ray equations for each pixel of each set of electric field lines and the camera's optical center are constructed, as illustrated in the diagram below. Figure 5 As shown.
[0051] From the above Take one of the groups Further obtain one of the pixels Then its corresponding pixel coordinates are Therefore, based on the camera intrinsic parameter matrix, the image plane coordinates can be obtained as follows: Its corresponding three-dimensional spatial coordinates in the camera coordinate system can be expressed as: ,in This refers to the camera's focal length. Based on the camera's extrinsic parameters, the image plane coordinates in the camera coordinate system are determined. Convert to world coordinate system Combined with the optical center position, i.e., the equipment position. The direction vector of the optically centered ray equation that can be obtained
[0052] in, Corresponding to Figure 3 The distances 1, 2, and 3 in the equation further express the optical center ray equation as follows: , This indicates the distance measured along the ray from the starting point.
[0053] Finally, the world coordinates of the electric field lines are calculated by solving the simultaneous equations of the camera optical center ray constructed from each set of electric field lines and their corresponding vertical plane equations. Transform the coordinate system to the vertical plane using the formula The complete world coordinates of the electric field lines can be obtained by performing least squares fitting.
[0054] Figure 6 The maximum and average errors of six sets of power lines connected by two towers spaced 400 meters apart are given for three different sag levels: 1 meter, 5 meters, and 10 meters. Under these conditions, as the sag of the power lines increases, both the average and maximum errors increase synchronously, but remain within the sub-meter error range. Figure 7 By observing the shape of the fitted electric field lines from different perspectives, it was demonstrated that the algorithm has superior fitting accuracy and that the algorithmic idea is feasible.
[0055] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art can make other variations or modifications based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.
Claims
1. A method for fitting power lines from surveillance images, characterized in that, Includes the following steps: S10: Construct a power line monitoring image dataset that can be used for semantic segmentation network learning; S20: Manually sample power line templates from real images, and perform grayscale multi-distribution processing on the templates to generate power line masks; S30: A synthetic power line monitoring image dataset was used to train the semantic segmentation network Unet; S40: Grouping the power line pixels segmented by the Unet network based on the region growing algorithm; S50: Construct the equation of the vertical plane containing each power line based on predefined engineering parameters; S60: The complete world coordinates of the electric field lines are obtained by combining the camera optical center ray equation constructed with the corresponding vertical plane equation.
2. The method for constructing a power line monitoring image dataset according to claim 1, characterized in that, Power line templates are manually sampled from real images, power line masks are obtained through data augmentation, and then combined with monitoring background images to synthesize a power line segmentation dataset.
3. The method for obtaining grouped pixels of power lines according to claim 1, characterized in that, By using a deep estimation network, after a certain training period, the network learns the scene of power transmission lines and surveillance images, enabling it to output accurate power line segmentation results.
4. The method for obtaining complete world coordinates of electric field lines by constructing the camera optical center ray equation based on electric field lines and the corresponding vertical plane equation according to claim 1, characterized in that, The equation of the vertical plane where each electric field line is located can be obtained by using predefined engineering parameters. Then, by combining the equation of the camera optical center ray constructed from the electric field lines, the complete world coordinates of the electric field lines can be obtained.