A method and related device for detecting and warning the tilt angle of a telegraph pole
By employing a multi-layer channel feature extraction network, dilated convolution, and channel attention module to create a utility pole image segmentation model, combined with an epipolar matching algorithm, the problems of low accuracy and high cost in utility pole detection are solved. This model achieves efficient and accurate tilt detection and early warning, ensuring the safety of power poles.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ZHONGSHAN POWER SUPPLY BUREAU OF GUANGDONG POWER GRID
- Filing Date
- 2025-06-30
- Publication Date
- 2026-05-29
AI Technical Summary
Existing technologies and traditional methods for detecting utility poles suffer from problems such as low efficiency, low accuracy, complex operation, high cost, and poor adaptability.
A utility pole image segmentation model is developed using a multi-layer channel feature extraction network MCFNet, a dilated convolutional strip pooling module DCSP, and a channel attention module CE. The parallax of the left and right side edges of the utility pole is calculated by combining the epipolar matching algorithm to construct a 3D point cloud. The tilt angle is determined by fitting the direction vectors of the left and right side edges of the utility pole and an early warning is issued.
It enables efficient and accurate detection of power pole tilt angles, reduces equipment costs and maintenance burden, improves detection accuracy and real-time performance, and ensures the safe operation of power poles.
Smart Images
Figure CN120726571B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of utility pole condition detection, and more specifically, to a method and related equipment for detecting and warning of utility pole tilt angle. Background Technology
[0002] Transmission towers, serving as the supporting hubs of transmission lines, are widely distributed throughout the power grid, undertaking the critical task of power transmission. However, due to factors such as soil erosion, soft soil, natural disasters, and external construction forces, transmission towers are prone to tilting. If tilting is not detected in time, it may lead to large-scale power outages or even endanger personal safety. Therefore, there is an urgent need for efficient and accurate tilt detection and early warning methods.
[0003] Traditional testing methods mainly include plumb line method, theodolite method, and total station prism-free method. The plumb line method requires manual climbing to the top of the pole to release the plumb bob, which is risky and inefficient. The theodolite method relies on a manual prism and multiple station setups, and is significantly affected by human error. Although the total station prism-free method does not require contact with the object being measured, it requires CAD drawing, which is complex and lacks real-time capability.
[0004] In recent years, technologies based on drone aerial photography and image detection models have gradually emerged. However, drone equipment has high procurement and maintenance costs, limited battery life, and cannot continuously monitor towers. Moreover, in complex environments, factors such as terrain interference, changes in light, and tower angle deflection in images greatly affect detection accuracy, significantly reducing the reliability of detection results.
[0005] Based on this, this application provides a pole tilt angle detection and early warning scheme, which avoids the defects of existing tilt angle detection methods and effectively ensures the safe operation of transmission towers. Summary of the Invention
[0006] This application provides a method and related equipment for detecting and warning the tilt angle of power poles, which effectively overcomes the shortcomings of the prior art, achieves accurate and efficient detection of the tilt angle of power poles, provides real-time warning, reduces operational risks, improves calculation accuracy, and strongly ensures the safe operation of transmission towers.
[0007] A method for detecting and warning of the tilt angle of utility poles includes:
[0008] Acquire images of utility poles captured by a camera and correct the images to obtain corrected images of the utility poles;
[0009] The corrected image of the utility pole is input into the trained utility pole image segmentation model to obtain a segmented image of the utility pole. The utility pole image segmentation model includes a multi-layer channel feature extraction network MCFNet, a dilated convolutional bar pooling module DCSP, and a channel attention module CE. The multi-layer channel feature extraction network MCFNet extracts semantic information of the feature map layer by layer from the channel aspect. The dilated convolutional bar pooling module DCSP improves the semantic segmentation effect of the image through dilated convolution and bar pooling. The channel attention module CE uses coordinate attention mechanism and channel attention mechanism to suppress redundant channel information.
[0010] The parallax of the left and right side edges of the utility pole in the segmented image of the utility pole is calculated using the epipolar matching algorithm, and a three-dimensional point cloud of the left and right side edges of the utility pole is constructed by combining the relationship between parallax and depth.
[0011] By fitting the direction vectors of the left and right sides of the utility pole in the three-dimensional point cloud, the tilt angle of the utility pole is determined. Combined with the preset threshold, tilt angle early warning detection is performed and an abnormal early warning signal is output.
[0012] Optionally, the multi-channel feature extraction network MCFNet includes multiple stacked deep channel feature extraction modules, wherein the processing of the input feature map by each deep channel feature extraction module includes:
[0013] The input feature map is compressed by using a convolutional layer with a kernel size of 1×1 to compress the number of channels, resulting in a compressed feature map.
[0014] The compressed feature map is input into a depthwise separable convolutional layer and an average pooling layer respectively to obtain two sub-feature maps;
[0015] The two sub-feature maps are concatenated and then convolved through a convolutional layer with a kernel size of 1×1 to output a depth-extracted feature map.
[0016] Optionally, the processing of the input feature map by the dilated convolutional bar pooling module DCSP includes:
[0017] The input feature map is convolved by a convolutional layer with a kernel size of 1×1 to generate a convolutional feature map.
[0018] The input feature map is convolved by multiple dilated convolutional layers with different dilation rates to generate multi-scale feature maps.
[0019] Perform bar pooling on the input feature map to generate a pooled feature map;
[0020] The convolutional feature map, the pooling feature map, and the multi-scale feature map are concatenated, and a fused dilated convolutional feature map is output through convolution.
[0021] Optionally, the channel attention module (CE) processes the input feature map, including:
[0022] The input feature map is processed using both coordinate attention and channel attention mechanisms to calculate the channel weights, resulting in the first channel weight and the second channel weight.
[0023] The weights of the first channel and the second channel are added together and activated by the Sigmoid function. The weighted feature map is then multiplied by the input feature map and output as a weighted feature map.
[0024] Optionally, the step of calculating the disparity of the left and right side edges of the utility pole in the segmented image using the epipolar matching algorithm, and constructing a 3D point cloud of the left and right side edges of the utility pole by combining the disparity and depth relationship, includes:
[0025] The segmented image of the utility pole is divided into multiple small regions. For each small region, the region-based epipolar matching algorithm is used to determine the matching points of the corresponding regions on the left and right side edges of the utility pole. The disparity of the left and right side edges of the utility pole is obtained by calculating the pixel coordinate difference of the matching points.
[0026] By combining the geometric relationship between parallax and depth, the parallax of the left and right side edges of the utility pole is converted into the depth values of each point on the left and right side edges of the utility pole, generating a side edge feature depth map.
[0027] Based on the side edge feature depth map, the coordinates of each point are expanded from two dimensions to three dimensions to construct a three-dimensional point cloud of the left and right side edges of the utility pole.
[0028] Optionally, determining the tilt angle of the utility pole by fitting the direction vectors of the left and right side edges of the utility pole in the three-dimensional point cloud of the left and right side edges includes:
[0029] Linear fitting is performed on the three-dimensional point cloud of the left and right sides of the utility pole, and the direction vectors of the left and right sides of the utility pole are obtained by combining the singular value decomposition method.
[0030] The tilt angle of the utility pole is determined based on the average lateral offset and height value of the left edge direction vector and the right edge direction vector.
[0031] A utility pole tilt angle detection and early warning device, comprising:
[0032] An image acquisition unit is used to acquire images of utility poles captured by a camera and to correct the utility pole images to obtain corrected utility pole images.
[0033] The segmentation model unit is used to input the corrected image of the utility pole into the trained utility pole image segmentation model to obtain a segmented image of the utility pole. The utility pole image segmentation model includes a multi-layer channel feature extraction network MCFNet, a dilated convolutional bar pooling module DCSP, and a channel attention module CE. The multi-layer channel feature extraction network MCFNet extracts semantic information of the feature map layer by layer from the channel aspect. The dilated convolutional bar pooling module DCSP improves the semantic segmentation effect of the image through dilated convolution and bar pooling. The channel attention module CE uses coordinate attention mechanism and channel attention mechanism to suppress redundant channel information.
[0034] A 3D point cloud unit is used to calculate the parallax of the left and right side edges of the utility pole in the segmented image of the utility pole using an epipolar matching algorithm, and to construct a 3D point cloud of the left and right side edges of the utility pole by combining the relationship between parallax and depth.
[0035] The tilt detection unit is used to determine the tilt angle of the utility pole by fitting the direction vectors of the left and right side edges of the utility pole in the three-dimensional point cloud of the left and right side edges, and to perform tilt angle early warning detection and output an abnormal early warning signal in combination with the preset threshold.
[0036] A device for detecting and warning the tilt angle of utility poles, comprising a memory and a processor;
[0037] The memory is used to store programs;
[0038] The processor is used to execute the program to implement the various steps of the pole tilt angle detection and early warning method as described in any of the above claims.
[0039] A readable storage medium having a computer program stored thereon, characterized in that, when the computer program is executed by a processor, it implements the various steps of the pole tilt angle detection and early warning method as described in any of the preceding claims.
[0040] A computer program product includes a computer program, characterized in that the computer program, when run by a processor, executes the various steps of the pole tilt angle detection and early warning method as described in any of the preceding claims.
[0041] As can be seen from the above technical solutions, the utility pole tilt angle detection and early warning method and related equipment provided in this application have many significant beneficial effects and can effectively avoid many defects of the prior art. In the image acquisition stage, the acquired utility pole images are corrected, avoiding the risks of manual high-altitude operations in the plumb line method and improving detection efficiency; at the same time, compared with drone aerial photography, it reduces equipment costs and maintenance burden, and overcomes the monitoring interruption problem caused by insufficient battery life. The correction operation can also eliminate image distortion caused by shooting angle and equipment errors, ensuring data accuracy and solving the problem of errors easily generated by manual operation in the theodolite method.
[0042] A utility pole image segmentation model, integrating the multi-channel feature extraction network MCFNet, the dilated convolutional bar pooling module DCSP, and the channel attention module CE, achieves efficient and accurate image segmentation. The MCFNet extracts semantic information from feature maps layer by layer at the channel level, reducing computational parameters while maintaining feature extraction quality and significantly improving computational efficiency. The dilated convolutional bar pooling module expands the receptive field using dilated convolution, improving the recognition of utility pole shape features. The CE module employs coordinate attention and channel attention mechanisms to effectively suppress redundant channel information and highlight relevant features related to the utility pole. This model eliminates the need for complex mapping processes, simplifying operation and improving real-time performance. Furthermore, it can handle complex environmental conditions such as terrain interference, lighting changes, and pole angle deflection in images, overcoming the shortcomings of existing image detection models, including low accuracy and poor reliability.
[0043] Furthermore, the epipolar matching algorithm is used to calculate the parallax of the left and right side edges of the utility pole in the segmented image, and a 3D point cloud of the left and right side edges of the utility pole is constructed. Compared with traditional 2D analysis, this method can more comprehensively reflect the actual state of the utility pole, avoid environmental and terrain interference, and significantly improve the accuracy of tilt angle calculation.
[0044] This application features real-time capability. Once the tilt angle of a power pole is detected to exceed the preset safety threshold, an early warning signal can be issued within seconds, allowing relevant personnel to promptly grasp the abnormal situation of the power pole and take swift measures to effectively ensure the safe operation of the transmission tower. This greatly reduces the probability of accidents caused by tower tilt and provides a solid and reliable technical guarantee for the stable operation of the power transmission network. Attached Figure Description
[0045] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.
[0046] Figure 1 This is a flowchart of a method for detecting and warning the tilt angle of a utility pole, as disclosed in an embodiment of this application.
[0047] Figure 2 This is a schematic diagram of the structure of a utility pole image segmentation model disclosed in an embodiment of this application;
[0048] Figure 3 This is a schematic diagram of the left and right side edge direction vectors disclosed in an embodiment of this application;
[0049] Figure 4This is a schematic diagram of the structure of a multi-channel feature extraction network MCFNet disclosed in an embodiment of this application;
[0050] Figure 5 This is a schematic diagram of the structure of a depth channel feature extraction module DF-G disclosed in an embodiment of this application;
[0051] Figure 6 This is a schematic diagram of the structure of a dilated convolutional strip pooling module (DCSP) disclosed in an embodiment of this application.
[0052] Figure 7 This is a schematic diagram of the structure of a channel attention module (CE) disclosed in an embodiment of this application;
[0053] Figure 8 This is a schematic diagram of a utility pole tilt angle detection and early warning device disclosed in an embodiment of this application;
[0054] Figure 9 This is a hardware structure block diagram of a utility pole tilt angle detection and early warning device disclosed in an embodiment of this application. Detailed Implementation
[0055] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0056] This application can be used in a wide variety of general-purpose or special-purpose computing device environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor devices, distributed computing environments including any of the above devices, etc.
[0057] The following section introduces the solution proposed in this application. The technical solution is as follows, and details are provided below.
[0058] Figure 1 This is a flowchart of a method for detecting and warning the tilt angle of utility poles, as disclosed in an embodiment of this application.
[0059] like Figure 1 As shown, the method may include:
[0060] Step S1: Acquire images of utility poles captured by a camera and correct the images to obtain corrected images of the utility poles.
[0061] Specifically, image acquisition and correction are crucial in the preliminary work for utility pole inspection. Step S1 mainly revolves around image acquisition and correction. First, various imaging devices, such as parallel binocular cameras, are used to acquire images of the utility pole. These devices can capture image information of the utility pole in different environments. However, due to factors such as shooting angle and equipment errors, the acquired images may have distortion and other problems, making them unsuitable for direct and accurate analysis. Therefore, the acquired utility pole images need to be corrected. Through specific algorithms and techniques, distortion factors in the image are eliminated, allowing the image to reflect the true shape of the utility pole, thus obtaining the corrected utility pole image.
[0062] Step S2: Input the corrected image of the utility pole into the trained utility pole image segmentation model to obtain a segmented image of the utility pole.
[0063] Specifically, the utility pole image segmentation model includes a multi-channel feature extraction network MCFNet, a dilated convolutional bar pooling module DCSP, and a channel attention module CE. The multi-channel feature extraction network MCFNet extracts semantic information from the feature map layer by layer from the channel aspect. The dilated convolutional bar pooling module DCSP improves the semantic segmentation effect of the image through dilated convolution and bar pooling. The channel attention module CE uses coordinate attention mechanism and channel attention mechanism to suppress redundant channel information.
[0064] The corrected image of the utility pole obtained in step S1 is input into a pre-trained utility pole image segmentation model. This model comprises a multi-channel feature extraction network (MCFNet), a dilated convolutional bar pooling module (DCSP), and a channel attention module (CE). The MCFNet extracts semantic information from the feature maps layer by layer at the channel level, ensuring feature extraction quality while reducing computational parameters and improving computational efficiency. The DCSP expands the model's receptive field using dilated convolution, enabling it to better capture the shape features of the utility pole. Combined with bar pooling, this further enhances the semantic segmentation effect. The CE uses coordinate attention and channel attention mechanisms to filter the input feature information, suppressing redundant channel information and highlighting effective features relevant to the utility pole. Through the collaborative processing of these modules, the final output is a segmented image of the utility pole, accurately separating it from the background for subsequent analysis and judgment of its state.
[0065] like Figure 2The diagram shows the structure of a utility pole image segmentation model. The utility pole image D1 is input into the model. D1 passes through a 3×3 convolutional layer (Conv) with a stride of 2 in the MCFNet multi-channel feature extraction network to obtain feature map D2. Then, D2 passes through the first depth channel feature extraction module (DF-G) to obtain feature map D3. D3 passes through the second DF-G module to obtain feature map D4. D4 passes through the third DF-G module to obtain feature map D5. D5 passes through a 1×1 convolutional layer (Conv) to obtain feature map D6. D6 passes through a dilated convolutional strip pooling module (DCSP) to obtain feature map D7. D7 is then upsampled by 4 times by the channel attention module (CE) to obtain feature map D8. Simultaneously, D3 passes through the channel attention module CE to obtain feature map D9. Finally, D8 and D9 are concatenated to obtain feature map D10. Finally, after passing through a Conv layer with a kernel size of 3×3, D10 undergoes a 4x upsampling operation to output the segmented image of the utility pole.
[0066] Step S3: Calculate the parallax of the left and right side edges of the utility pole in the segmented image of the utility pole using the epipolar matching algorithm, and construct a three-dimensional point cloud of the left and right side edges of the utility pole by combining the relationship between parallax and depth.
[0067] Specifically, the first step is to segment the image and calculate disparity. The image of the utility pole is divided into multiple small regions, and a region-based epipolar matching algorithm is applied to each region. This algorithm can find matching points for corresponding regions on the left and right edges of the utility pole. By calculating the difference in pixel coordinates of these matching points, the disparity of the left and right edges of the utility pole can be obtained. This method can determine the disparity information at different locations with relatively high accuracy.
[0068] Next comes the depth value conversion. Based on the established geometric relationship between disparity and depth, the disparity of the left and right side edges of the utility pole, calculated earlier, is further converted into depth values for each point on the left and right side edges, thus generating a side edge feature depth map. The depth map visually displays the depth dimension information of each point on the side edges of the utility pole.
[0069] Finally, a 3D point cloud is constructed. Based on the side edge feature depth map, the original two-dimensional coordinates of each point are expanded into three-dimensional coordinates, completing the transformation from planar information to three-dimensional information, and ultimately constructing the 3D point cloud of the left and right side edges of the utility pole. The 3D point cloud can more comprehensively and intuitively present the spatial morphology of the utility pole's side edges, providing a richer and more accurate data foundation for subsequent analysis of the utility pole's actual condition, such as calculating the tilt angle.
[0070] In other words, the process of calculating the disparity of the left and right side edges of the utility pole in the segmented image using the epipolar matching algorithm, and constructing the 3D point cloud of the left and right side edges of the utility pole by combining the disparity and depth relationship, specifically includes:
[0071] ① Divide the segmented image of the utility pole into multiple small regions. For each small region, use the region-based epipolar matching algorithm to determine the matching points of the corresponding regions on the left and right side edges of the utility pole. And calculate the disparity of the left and right side edges of the utility pole by calculating the pixel coordinate difference of the matching points.
[0072] ② Combining the geometric relationship between parallax and depth, the parallax of the left and right side edges of the utility pole is converted into the depth values of each point on the left and right side edges of the utility pole, generating a side edge feature depth map;
[0073] ③ Based on the side edge feature depth map, the coordinates of each point are expanded from two dimensions to three dimensions to construct a three-dimensional point cloud of the left and right side edges of the utility pole.
[0074] Step S4: By fitting the direction vectors of the left and right sides of the utility pole in the three-dimensional point cloud, the tilt angle of the utility pole is determined, and the tilt angle is detected and an abnormal warning signal is output in combination with the preset threshold.
[0075] Specifically, in determining the tilt angle, the first step is to perform linear fitting on the 3D point cloud of the left and right sides of the utility pole. Linear fitting can fit the 3D point cloud data into a straight line, allowing for better analysis of its directional characteristics. Then, singular value decomposition is used, which can accurately obtain the direction vectors of the left and right sides of the utility pole. These two vectors represent the orientation of the left and right sides of the utility pole in space.
[0076] Next, based on the obtained left and right edge direction vectors, their average lateral offset is calculated, which represents the relative displacement of the left and right edges in the horizontal direction. This is then combined with the height value, which is the vertical information of the utility pole. Through comprehensive analysis of these data, the tilt angle of the utility pole can be determined, reflecting the degree to which the pole deviates from its vertical position.
[0077] In terms of early warning detection, the calculated tilt angle of the utility pole is compared with a preset threshold. The preset threshold is an angle limit set in advance based on factors such as utility pole safety standards. When the detected tilt angle exceeds the preset threshold, it indicates that the utility pole is in an unsafe tilted state. At this time, the system will output an abnormal warning signal to remind relevant personnel to take timely measures to avoid safety accidents caused by the tilted utility pole.
[0078] The process of determining the tilt angle of the utility pole by fitting the direction vectors of the left and right side edges of the utility pole in the three-dimensional point cloud can specifically include:
[0079] ① Perform linear fitting on the three-dimensional point cloud of the left and right sides of the utility pole, and combine the singular value decomposition method to obtain the direction vector of the left side edge and the direction vector of the right side edge of the utility pole;
[0080] ② Determine the tilt angle of the utility pole based on the average lateral offset and height value of the left edge direction vector and the right edge direction vector.
[0081] Specifically, a straight line is fitted to the 3D point cloud of the side edge of the utility pole. Using the fitted straight line, the left camera is set to the same spatial coordinate system. By combining the singular value decomposition method, the direction vector of the fitted straight line is obtained, which is the direction vector of the left edge of the utility pole. Repeat the above steps to obtain the direction vector of the right edge. Left edge direction vector and the direction vector of the right edge like Figure 3 As shown. For ease of calculation, the direction vectors of the left and right edges can be taken to be the same. The formula for calculating the tilt angle of a utility pole is as follows:
[0082]
[0083]
[0084] In the formula, The tilt of the utility pole The angle of inclination of the utility pole.
[0085] In some embodiments of this application, combined with Figure 4 and Figure 5 An introduction to the multi-channel feature extraction network MCFNet will be provided, which may include:
[0086] The multi-channel feature extraction network MCFNet includes multiple stacked deep channel feature extraction modules, wherein the processing of the input feature map by each deep channel feature extraction module includes:
[0087] The input feature map is compressed by using a convolutional layer with a kernel size of 1×1 to compress the number of channels, resulting in a compressed feature map.
[0088] The compressed feature map is input into a depthwise separable convolutional layer and an average pooling layer respectively to obtain two sub-feature maps;
[0089] The two sub-feature maps are concatenated and then convolved through a convolutional layer with a kernel size of 1×1 to output a depth-extracted feature map.
[0090] Specifically, the multi-channel feature extraction network MCFNet plays a key role in the task of segmenting utility pole images. It extracts the semantic information of the feature map of the image layer by layer through the stacking of multiple deep channel feature extraction modules (DF-G modules) to achieve accurate segmentation of utility poles.
[0091] like Figure 4 As shown, when processing utility pole images, the multi-channel feature extraction network MCFNet first inputs the original utility pole image into a convolutional layer Conv. This convolutional layer can use a 3×3 kernel with a stride of 2. Through this step, the image is initially converted into a feature map, the purpose of which is to extract some basic features of the image and to downsample the image to a certain extent, making subsequent processing more efficient.
[0092] Next, we move to the DF-G module. Each DF-G module performs a series of ordered operations on the input feature map. The first step is channel compression, which uses a 1×1 convolutional layer to process the input feature map. The purpose of this operation is to reduce the number of channels in the feature map while preserving as much key information as possible, thereby reducing the complexity of subsequent calculations.
[0093] The compressed feature map obtained after channel compression is fed into two different paths. One path is a depthwise separable convolutional layer, which uses a 3×3 depthwise separable convolution with a stride of 2. Depthwise separable convolution is an optimized convolution method that decomposes standard convolution into depthwise convolution and pointwise convolution, significantly reducing computation while maintaining feature extraction capabilities. Through depthwise separable convolution, spatial and channel features in the feature map are further extracted, and downsampling is performed again. The other path is an average pooling layer, which downsamples the feature map while retaining some global information.
[0094] The two sub-feature maps output from the depthwise separable convolutional layer and the average pooling layer are concatenated. This concatenation operation integrates feature information from different paths, enriching the feature representation. The concatenated feature map is then processed by a 1×1 convolutional layer. This step mainly integrates the concatenated features and performs further feature extraction, ultimately outputting a depthwise extracted feature map.
[0095] These deeply extracted feature maps are sequentially fed into subsequent DF-G modules, repeating the above operation to continuously extract higher-level, more semantically informative features. As features are passed and processed across multiple DF-G modules, the utility pole-related features in the image are gradually enhanced and refined. After processing by the final DF-G module, the feature map is passed through a 1×1 Conv layer to obtain the final output feature map of MCFNet. This output feature map contains rich and accurate semantic information about the utility poles, providing strong support for accurate segmentation of utility poles in subsequent images.
[0096] The specific execution flow of the multi-channel feature extraction network MCFNet is as follows: The input utility pole image D1 is convolved to obtain feature map D2. D2 is then passed through the first DF-G module to obtain feature map D3. D3 is then passed through the second DF-G module to obtain feature map D4. D4 is then passed through the third DF-G module to obtain feature map D5. Finally, D5 is passed through a Conv layer with a convolution kernel size of 1×1 to obtain feature map D6, which serves as the output of MCFNet.
[0097] like Figure 5 As shown, for each DF-G module, taking the input of D2 into the first DF-G module as an example: First, the input feature map D2 is compressed by a convolutional layer with a kernel size of 1×1 to obtain a compressed feature map G1; then, G1 is input into a depthwise separable convolutional layer DSConv with a kernel size of 3×3 and a stride of 2 and an average pooling layer AvgPool to obtain sub-feature maps G2 and G3 respectively; then, G2 and G3 are concatenated to obtain feature map G4; finally, G4 is convolved by a 1×1 convolutional layer to output the depthwise extracted feature map D3.
[0098] D3 is then fed into subsequent DF-G modules for similar processing. After passing through the second and third DF-G modules, D4 and D5 are obtained respectively. D5 then passes through a 1×1 convolutional layer to obtain feature map D6, which serves as the final output of MCFNet. By extracting semantic information from feature maps layer by layer in this way, MCFNet can better assist in the accurate segmentation of utility pole images.
[0099] In some embodiments of this application, combined with Figure 6 The processing procedure of the dilated convolutional strip pooling module DCSP is described, which may include:
[0100] The input feature map is convolved by a convolutional layer with a kernel size of 1×1 to generate a convolutional feature map.
[0101] The input feature map is convolved by multiple dilated convolutional layers with different dilation rates to generate multi-scale feature maps.
[0102] Perform bar pooling on the input feature map to generate a pooled feature map;
[0103] The convolutional feature map, the pooling feature map, and the multi-scale feature map are concatenated, and a fused dilated convolutional feature map is output through convolution.
[0104] Specifically, such as Figure 6 As shown, feature map D6 is first taken as input. The DCSP module uses multiple dilated convolutional layers with different dilation rates to convolve the input feature map D6. It is then processed by a 1×1 convolutional layer Conv to obtain the convolutional feature map C1. This step allows adjustment of the number of channels in the input feature map and extracts some preliminary features.
[0105] Subsequently, dilated convolutional layers DConv with kernel sizes of 3×3 and dilation rates of 6, 12, and 18 were used to obtain feature maps C2, C3, and C4, respectively. Dilated convolution, by introducing a dilation rate, expands the receptive field of the convolutional kernel without increasing the number of parameters or computational cost. This allows it to capture information at different scales in the image, generating multi-scale feature maps, and enabling the model to better understand the shape and structure of objects in the image.
[0106] Simultaneously, bar pooling is performed on the input feature map D6, resulting in feature map C5 through the bar pooling layer SPPool. Bar pooling can pool feature maps in a specific direction, which helps to extract long bar or linear features from the image and further enrich the feature representation.
[0107] Then, the obtained feature maps C1, C2, C3, C4, and C5 are concatenated to obtain feature map C6. The concatenation operation merges feature maps generated by different paths, integrating multi-scale and different types of feature information.
[0108] Finally, the stitched feature map C6 is further convolved through a Conv layer with a kernel size of 1×1, outputting feature map D7, which is the fused dilated convolutional feature map. This step further refines and integrates the fused features, making the final output feature map contain more comprehensive and discriminative information, thereby improving the accuracy of image semantic segmentation and better identifying and segmenting target objects such as utility poles and towers in the image.
[0109] In some embodiments of this application, combined with Figure 7 The processing procedure of the Channel Attention Module (CE) is described, which may include:
[0110] The input feature map is processed using both coordinate attention and channel attention mechanisms to calculate the channel weights, resulting in the first channel weight and the second channel weight.
[0111] The weights of the first channel and the second channel are added together and activated by the Sigmoid function. The weighted feature map is then multiplied by the input feature map and output as a weighted feature map.
[0112] Specifically, the Channel Attention Module (CE) aims to suppress redundant channel information and highlight effective feature information. Its processing procedure is as follows:
[0113] First, the feature map D7 is taken as input. The module employs two different channel attention mechanisms: coordinate attention (CA) and channel attention (ECA). Coordinate attention (CA) is primarily used to extract the correlation between different channels in image features, enabling the network to focus on important feature information. Channel attention (ECA), on the other hand, is an efficient channel attention mechanism for deep convolutional neural networks, enhancing the model's focus on feature channels. These two mechanisms are used to calculate two channel attention weights on the input feature map D7: the first channel weight W1 and the second channel weight W2.
[0114] Next, the obtained first channel weight W1 and second channel weight W2 are added together, and then activated by the Sigmoid function. The Sigmoid function maps the added weight values to a range of 0 to 1, resulting in a probabilistic weight distribution to measure the importance of each channel.
[0115] Finally, the weights activated by the Sigmoid function are multiplied by the corresponding elements of the input feature map D7, and a weighting operation is performed to obtain feature map D8, which is the weighted feature map. The calculation formula is as follows:
[0116]
[0117] In the formula, D7 represents the input feature map, D8 represents the output feature map, CA(.) represents the coordinate attention mechanism, and ECA(.) represents the channel attention mechanism. This represents the Sigmoid function, where W1 and W2 are the channel weights calculated by the coordinate attention mechanism and the channel attention mechanism, respectively. This indicates that corresponding elements in the feature map are multiplied together.
[0118] By doing so, the CE module can adjust the input feature map according to the importance of the channels, suppressing those channels that contain redundant information and highlighting the effective feature information channels that are important for tasks such as power pole tower image segmentation, thereby improving the model's performance and segmentation accuracy.
[0119] The following describes a pole tilt angle detection and early warning device provided in the embodiments of this application. The pole tilt angle detection and early warning device described below can be referred to in correspondence with the pole tilt angle detection and early warning method described above.
[0120] See Figure 8 , Figure 8 This is a schematic diagram of a pole tilt angle detection and early warning device disclosed in an embodiment of this application.
[0121] like Figure 8 As shown, the utility pole tilt angle detection and early warning device may include:
[0122] The image acquisition unit 110 is used to acquire images of utility poles captured by the shooting device, and to correct the utility pole images to obtain corrected images of utility poles;
[0123] The segmentation model unit 120 is used to input the corrected image of the utility pole into the trained utility pole image segmentation model to obtain a segmented image of the utility pole. The utility pole image segmentation model includes a multi-layer channel feature extraction network MCFNet, a dilated convolutional bar pooling module DCSP, and a channel attention module CE. The multi-layer channel feature extraction network MCFNet extracts semantic information of the feature map layer by layer from the channel aspect. The dilated convolutional bar pooling module DCSP improves the semantic segmentation effect of the image through dilated convolution and bar pooling. The channel attention module CE uses coordinate attention mechanism and channel attention mechanism to suppress redundant channel information.
[0124] The three-dimensional point cloud unit 130 is used to calculate the parallax of the left and right side edges of the utility pole in the segmented image of the utility pole using the epipolar matching algorithm, and to construct the three-dimensional point cloud of the left and right side edges of the utility pole by combining the relationship between parallax and depth.
[0125] The tilt detection unit 140 is used to determine the tilt angle of the utility pole by fitting the direction vectors of the left and right side edges of the utility pole in the three-dimensional point cloud of the left and right side edges of the utility pole, and to perform tilt angle early warning detection and output an abnormal early warning signal in combination with the preset threshold.
[0126] As can be seen from the above technical solutions, the utility pole tilt angle detection and early warning method and related equipment provided in this application have many significant beneficial effects and can effectively avoid many defects of the prior art. In the image acquisition stage, the acquired utility pole images are corrected, avoiding the risks of manual high-altitude operations in the plumb line method and improving detection efficiency; at the same time, compared with drone aerial photography, it reduces equipment costs and maintenance burden, and overcomes the monitoring interruption problem caused by insufficient battery life. The correction operation can also eliminate image distortion caused by shooting angle and equipment errors, ensuring data accuracy and solving the problem of errors easily generated by manual operation in the theodolite method.
[0127] A utility pole image segmentation model, integrating the multi-channel feature extraction network MCFNet, the dilated convolutional bar pooling module DCSP, and the channel attention module CE, achieves efficient and accurate image segmentation. The MCFNet extracts semantic information from feature maps layer by layer at the channel level, reducing computational parameters while maintaining feature extraction quality and significantly improving computational efficiency. The dilated convolutional bar pooling module expands the receptive field using dilated convolution, improving the recognition of utility pole shape features. The CE module employs coordinate attention and channel attention mechanisms to effectively suppress redundant channel information and highlight relevant features related to the utility pole. This model eliminates the need for complex mapping processes, simplifying operation and improving real-time performance. Furthermore, it can handle complex environmental conditions such as terrain interference, lighting changes, and pole angle deflection in images, overcoming the shortcomings of existing image detection models, including low accuracy and poor reliability.
[0128] Furthermore, the epipolar matching algorithm is used to calculate the parallax of the left and right side edges of the utility pole in the segmented image, and a 3D point cloud of the left and right side edges of the utility pole is constructed. Compared with traditional 2D analysis, this method can more comprehensively reflect the actual state of the utility pole, avoid environmental and terrain interference, and significantly improve the accuracy of tilt angle calculation.
[0129] This application features real-time capability. Once the tilt angle of a power pole is detected to exceed the preset safety threshold, an early warning signal can be issued within seconds, allowing relevant personnel to promptly grasp the abnormal situation of the power pole and take swift measures to effectively ensure the safe operation of the transmission tower. This greatly reduces the probability of accidents caused by tower tilt and provides a solid and reliable technical guarantee for the stable operation of the power transmission network.
[0130] The pole tilt angle detection and early warning device provided in this application embodiment can be applied to pole tilt angle detection and early warning equipment. Figure 9 The hardware structure block diagram of the utility pole tilt angle detection and early warning device is shown. (Refer to...) Figure 9 The hardware structure of the utility pole tilt detection and early warning device may include: at least one processor 1, at least one communication interface 2, at least one memory 3 and at least one communication bus 4;
[0131] In this embodiment of the application, the number of processor 1, communication interface 2, memory 3, and communication bus 4 is at least one, and processor 1, communication interface 2, and memory 3 communicate with each other through communication bus 4;
[0132] Processor 1 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.
[0133] Memory 3 may include high-speed RAM, and may also include non-volatile memory, such as at least one disk storage device;
[0134] The memory stores a program, which the processor can call. The program is used for:
[0135] Acquire images of utility poles captured by a camera and correct the images to obtain corrected images of the utility poles;
[0136] The corrected image of the utility pole is input into the trained utility pole image segmentation model to obtain a segmented image of the utility pole. The utility pole image segmentation model includes a multi-layer channel feature extraction network MCFNet, a dilated convolutional bar pooling module DCSP, and a channel attention module CE. The multi-layer channel feature extraction network MCFNet extracts semantic information of the feature map layer by layer from the channel aspect. The dilated convolutional bar pooling module DCSP improves the semantic segmentation effect of the image through dilated convolution and bar pooling. The channel attention module CE uses coordinate attention mechanism and channel attention mechanism to suppress redundant channel information.
[0137] The parallax of the left and right side edges of the utility pole in the segmented image of the utility pole is calculated using the epipolar matching algorithm, and a three-dimensional point cloud of the left and right side edges of the utility pole is constructed by combining the relationship between parallax and depth.
[0138] By fitting the direction vectors of the left and right sides of the utility pole in the three-dimensional point cloud, the tilt angle of the utility pole is determined. Combined with the preset threshold, tilt angle early warning detection is performed and an abnormal early warning signal is output.
[0139] Optionally, the refined and extended functions of the program can be referred to the above description.
[0140] This application embodiment also provides a readable storage medium that can store a program suitable for execution by a processor, the program being used for:
[0141] Acquire images of utility poles captured by a camera and correct the images to obtain corrected images of the utility poles;
[0142] The corrected image of the utility pole is input into the trained utility pole image segmentation model to obtain a segmented image of the utility pole. The utility pole image segmentation model includes a multi-layer channel feature extraction network MCFNet, a dilated convolutional bar pooling module DCSP, and a channel attention module CE. The multi-layer channel feature extraction network MCFNet extracts semantic information of the feature map layer by layer from the channel aspect. The dilated convolutional bar pooling module DCSP improves the semantic segmentation effect of the image through dilated convolution and bar pooling. The channel attention module CE uses coordinate attention mechanism and channel attention mechanism to suppress redundant channel information.
[0143] The parallax of the left and right side edges of the utility pole in the segmented image of the utility pole is calculated using the epipolar matching algorithm, and a three-dimensional point cloud of the left and right side edges of the utility pole is constructed by combining the relationship between parallax and depth.
[0144] By fitting the direction vectors of the left and right sides of the utility pole in the three-dimensional point cloud, the tilt angle of the utility pole is determined. Combined with the preset threshold, tilt angle early warning detection is performed and an abnormal early warning signal is output.
[0145] Optionally, the refined and extended functions of the program can be referred to the above description.
[0146] This application also provides a computer program product, including a computer program, wherein the computer program is executed by a processor using the following method:
[0147] Acquire images of utility poles captured by a camera and correct the images to obtain corrected images of the utility poles;
[0148] The corrected image of the utility pole is input into the trained utility pole image segmentation model to obtain a segmented image of the utility pole. The utility pole image segmentation model includes a multi-layer channel feature extraction network MCFNet, a dilated convolutional bar pooling module DCSP, and a channel attention module CE. The multi-layer channel feature extraction network MCFNet extracts semantic information of the feature map layer by layer from the channel aspect. The dilated convolutional bar pooling module DCSP improves the semantic segmentation effect of the image through dilated convolution and bar pooling. The channel attention module CE uses coordinate attention mechanism and channel attention mechanism to suppress redundant channel information.
[0149] The parallax of the left and right side edges of the utility pole in the segmented image of the utility pole is calculated using the epipolar matching algorithm, and a three-dimensional point cloud of the left and right side edges of the utility pole is constructed by combining the relationship between parallax and depth.
[0150] By fitting the direction vectors of the left and right sides of the utility pole in the three-dimensional point cloud, the tilt angle of the utility pole is determined. Combined with the preset threshold, tilt angle early warning detection is performed and an abnormal early warning signal is output.
[0151] Optionally, the refined and extended functions of the program can be referred to the above description.
[0152] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0153] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.
[0154] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A method for detecting and warning of the tilt angle of utility poles, characterized in that, include: Acquire images of utility poles captured by a camera and correct the images to obtain corrected images of the utility poles; The corrected image of the utility pole is input into the trained utility pole image segmentation model to obtain a segmented image of the utility pole. The utility pole image segmentation model includes a multi-layer channel feature extraction network MCFNet, a dilated convolutional bar pooling module DCSP, and a channel attention module CE. The multi-layer channel feature extraction network MCFNet extracts semantic information of the feature map layer by layer from the channel aspect. The dilated convolutional bar pooling module DCSP improves the semantic segmentation effect of the image through dilated convolution and bar pooling. The channel attention module CE uses coordinate attention mechanism and channel attention mechanism to suppress redundant channel information. The parallax of the left and right side edges of the utility pole in the segmented image of the utility pole is calculated using the epipolar matching algorithm, and a three-dimensional point cloud of the left and right side edges of the utility pole is constructed by combining the relationship between parallax and depth. By fitting the direction vectors of the left and right sides of the utility pole in the three-dimensional point cloud, the tilt angle of the utility pole is determined, and tilt angle early warning detection is performed in combination with a preset threshold and an abnormal early warning signal is output. The multi-channel feature extraction network MCFNet includes multiple stacked deep channel feature extraction modules, wherein the processing of the input feature map by each deep channel feature extraction module includes: The input feature map is compressed by using a convolutional layer with a kernel size of 1×1 to compress the number of channels, resulting in a compressed feature map. The compressed feature map is input into a depthwise separable convolutional layer and an average pooling layer respectively to obtain two sub-feature maps; The two sub-feature maps are concatenated and then convolved through a convolutional layer with a kernel size of 1×1 to output a depth-extracted feature map.
2. The method according to claim 1, characterized in that, The dilated convolutional bar pooling module (DCSP) processes the input feature map, including: The input feature map is convolved by a convolutional layer with a kernel size of 1×1 to generate a convolutional feature map. The input feature map is convolved by multiple dilated convolutional layers with different dilation rates to generate multi-scale feature maps. Perform bar pooling on the input feature map to generate a pooled feature map; The convolutional feature map, the pooling feature map, and the multi-scale feature map are concatenated, and a fused dilated convolutional feature map is output through convolution.
3. The method according to claim 1, characterized in that, The channel attention module (CE) processes the input feature map, including: The input feature map is processed using both coordinate attention and channel attention mechanisms to calculate the channel weights, resulting in the first channel weight and the second channel weight. The weights of the first channel and the second channel are added together and activated by the Sigmoid function. The weighted feature map is then multiplied by the input feature map and output as a weighted feature map.
4. The method according to claim 1, characterized in that, The step of calculating the disparity of the left and right side edges of the utility pole in the segmented image using the epipolar matching algorithm, and constructing a 3D point cloud of the left and right side edges of the utility pole by combining the disparity and depth relationship, includes: The segmented image of the utility pole is divided into multiple small regions. For each small region, the region-based epipolar matching algorithm is used to determine the matching points of the corresponding regions on the left and right side edges of the utility pole. The disparity of the left and right side edges of the utility pole is obtained by calculating the pixel coordinate difference of the matching points. By combining the geometric relationship between parallax and depth, the parallax of the left and right side edges of the utility pole is converted into the depth values of each point on the left and right side edges of the utility pole, generating a side edge feature depth map. Based on the side edge feature depth map, the coordinates of each point are expanded from two dimensions to three dimensions to construct a three-dimensional point cloud of the left and right side edges of the utility pole.
5. The method according to claim 1, characterized in that, The process of determining the tilt angle of the utility pole by fitting the direction vectors of the left and right side edges of the utility pole in the three-dimensional point cloud includes: Linear fitting is performed on the three-dimensional point cloud of the left and right sides of the utility pole, and the direction vectors of the left and right sides of the utility pole are obtained by combining the singular value decomposition method. The tilt angle of the utility pole is determined based on the average lateral offset and height value of the left edge direction vector and the right edge direction vector.
6. A device for detecting and warning the tilt angle of utility poles, characterized in that, include: An image acquisition unit is used to acquire images of utility poles captured by a camera and to correct the utility pole images to obtain corrected utility pole images. The segmentation model unit is used to input the corrected image of the utility pole into the trained utility pole image segmentation model to obtain a segmented image of the utility pole. The utility pole image segmentation model includes a multi-layer channel feature extraction network MCFNet, a dilated convolutional bar pooling module DCSP, and a channel attention module CE. The multi-layer channel feature extraction network MCFNet extracts semantic information of the feature map layer by layer from the channel aspect. The dilated convolutional bar pooling module DCSP improves the semantic segmentation effect of the image through dilated convolution and bar pooling. The channel attention module CE uses coordinate attention mechanism and channel attention mechanism to suppress redundant channel information. A 3D point cloud unit is used to calculate the parallax of the left and right side edges of the utility pole in the segmented image of the utility pole using an epipolar matching algorithm, and to construct a 3D point cloud of the left and right side edges of the utility pole by combining the relationship between parallax and depth. The tilt detection unit is used to determine the tilt angle of the utility pole by fitting the direction vectors of the left and right side edges of the utility pole in the three-dimensional point cloud of the left and right side edges of the utility pole, and to perform tilt angle early warning detection and output an abnormal early warning signal in combination with a preset threshold. The multi-channel feature extraction network MCFNet includes multiple stacked deep channel feature extraction modules, wherein the processing of the input feature map by each deep channel feature extraction module includes: The input feature map is compressed by using a convolutional layer with a kernel size of 1×1 to compress the number of channels, resulting in a compressed feature map. The compressed feature map is input into a depthwise separable convolutional layer and an average pooling layer respectively to obtain two sub-feature maps; The two sub-feature maps are concatenated and then convolved through a convolutional layer with a kernel size of 1×1 to output a depth-extracted feature map.
7. A device for detecting and warning the tilt angle of utility poles, characterized in that, Including memory and processor; The memory is used to store programs; The processor is used to execute the program to implement each step of the pole tilt angle detection and early warning method as described in any one of claims 1-5.
8. A readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements each step of the pole tilt angle detection and early warning method as described in any one of claims 1-5.
9. A computer program product, comprising a computer program, characterized in that, The computer program is executed by the processor to perform each step of the pole tilt angle detection and early warning method as described in any one of claims 1-5.