Power grid transmission line tower inclination detection method based on remote sensing image semantic segmentation

Abstract

The invention discloses a power grid transmission line tower inclination detection method based on remote sensing image semantic segmentation, and relates to the field of electric power safety and artificial intelligence. Comprising the following steps: step 1), preprocessing a remote sensing image, including shadow detection and image denoising; 2) performing semantic segmentation on the remote sensing image through an improved U-Net neural network, and extracting a tower body area and a tower top area of the power grid transmission tower; and step 3), according to semantic segmentation results of the tower body and the tower top of the transmission tower, calculating center point coordinates of the two areas, and according to the two center point coordinates, calculating the inclination of the tower. According to the invention, by using the remote sensing image and semantic segmentation technology, the rapid detection of the inclination condition of the pole tower in a large-range area can be realized.

Description

Technical Field

[0001] The present invention relates to the fields of electric power safety and artificial intelligence, and in particular to a method for detecting the inclination of power transmission line towers based on remote sensing image semantic segmentation. Background Art

[0002] With the rapid development of the power industry and the continuous expansion of the power grid, transmission line towers are the key infrastructure for the operation of the power grid, and their stability and safety are crucial to the overall operation of the power system. However, during the long-term operation, transmission line towers are easily tilted due to the influence of natural factors (such as extreme weather, geological disasters, etc.) and human factors (such as construction disturbances, line aging, etc.), which may not only lead to a reduction in electrical safety distances, but also cause serious accidents such as line breaks, tripping, and tower collapse, posing a major threat to the stable operation of the power system and the safety of life and property.

[0003] In the field of power inspection, the traditional method of detecting the inclination of pole towers mainly relies on manual operation, using traditional measuring tools such as plumb bobs and theodolites. However, this method is not only labor-intensive and inefficient, but its measurement accuracy is also easily restricted by various external factors such as light changes and observation angles, thus affecting the accuracy and reliability of the detection. In recent years, an automated monitoring solution based on satellite positioning technology and wireless communication technology has emerged. This solution integrates the high-precision positioning capability of the Beidou satellite positioning system and combines the data of a variety of high-precision sensors to achieve remote and automated monitoring of the inclination state of the pole tower. Although these automated monitoring solutions have performed well in reducing manual intervention and improving monitoring efficiency, they also face some challenges. First, the installation of these special equipment requires a high initial investment cost; second, in complex environments, wireless communication signals may be interfered with, resulting in delays or interruptions in data transmission. Therefore, studying an efficient, accurate, and low-cost method for detecting the inclination of pole towers is of great significance for improving the safety and stability of power grid operation. Summary of the invention

[0004] Aiming at the shortcomings of traditional transmission line tower tilt detection methods, the present invention proposes a power grid transmission line tower tilt detection method based on remote sensing image semantic segmentation.

[0005] The technical solution of the present invention is: a method for detecting the inclination of power transmission line towers based on remote sensing image semantic segmentation, comprising the following steps:

[0006] Step 1): Preprocess the remote sensing image, including shadow detection and image denoising;

[0007] Step 2): Perform semantic segmentation on the remote sensing image through the improved U-Net neural network to extract the tower body and tower top area of ​​the power grid transmission tower;

[0008] Step 3): According to the semantic segmentation results of the transmission tower body and tower top, the center point coordinates of the two areas are calculated, and the inclination of the tower is calculated based on the coordinates of the two center points.

[0009] Shadow detection in step 1) includes:

[0010] Remote sensing images introduced into C 1 C 2 C 3 Color space, the original RGB color space is nonlinearly transformed according to equations (1) to (3);

[0011]

[0012]

[0013]

[0014] In the formula, R, G, and B represent the brightness values ​​of the red, green, and blue channels of each pixel in the image, respectively;

[0015] C-based 3 The channel performs shadow detection and defines the variance filter function to calculate the neighborhood variance of the pixel point, as shown in equations (4) and (5);

[0016]

[0017]

[0018] Where (i, j) represents the pixel position in the image, I represents the image pixel matrix, f(i, j) represents the pixel value, M represents the length of the image, and N represents the width of the image.

[0019] Image denoising in step 1) includes:

[0020] The SE channel attention mechanism is added on the basis of the DnCNN network. Before the SE module, the dimension of the feature map is H*W*C. The process of adding SE attention is as follows:

[0021] (1) First, perform spatial feature compression on the feature map and implement global average pooling in the spatial dimension to obtain a 1*1*C feature map;

[0022] (2) Through the fully connected layer FC learning, a feature map with channel attention is obtained, and its dimension is still 1*1*C; the feature map of channel attention 1*1*C and the feature map of the original input H*W*C are multiplied by the weight coefficient channel by channel, and finally the feature map with channel attention is output.

[0023] Step 2) includes:

[0024] 2.1) Build a U-Net neural network, including downsampling part and upsampling part;

[0025] 2.2) Add a CA module that takes into account both spatial attention and channel attention in the downsampling part of the U-Net neural network;

[0026] 2.3) Add a multi-scale feature extraction neural network before the U-Net downsampling part.

[0027] In step 2.2),

[0028] The CA module includes: coordinate information embedding and coordinate attention generation;

[0029] In the coordinate information embedding stage, the module first receives the feature map output by the previous convolutional layer as input; then, to capture the important spatial information along the length and width in the feature map, the module performs directional global pooling operations on the feature map respectively; the pooling results in these two directions are then concatenated along the channel dimension to form a tensor containing rich position information;

[0030] Entering the coordinate attention generation stage, the module first compresses and activates the spliced ​​coordinate information tensor; then, through splitting and independent convolution operations, the module restores the original number of channels of the feature map and generates two independent attention weight maps for the length and width directions respectively; these two weight maps are then normalized by the Sigmoid function as the attention coefficients in their respective directions;

[0031] Finally, the module performs matrix multiplication of the original feature map with the attention coefficients in these two directions.

[0032] In step 2.3),

[0033] A CNN neural network with multi-scale feature fusion is adopted, and the network structure is as follows: the input first passes through convolution layers with convolution kernels of (1×1, 3×3, 5×5), the results of different convolution layers are spliced ​​together, and then the output is obtained through two fully connected layers.

[0034] Step 3) includes:

[0035] 3.1) Based on the semantic segmentation results, calculate the coordinates of the center point A of the tower body and the center point B of the tower top respectively;

[0036] Use OpenCV's moments function to calculate moments, including zero-order moments (m 00 ) and first-order moment (m 10 ,m 01 ); where the zero-order moment represents the total number of pixels within the contour, that is, the area of ​​the contour; the first-order moment m 10is the weighted sum of the pixels of the contour along the x-axis, m 01 is the weighted sum of the pixels of the contour along the y-axis, and the center point coordinates are calculated using formulas (6) and (7):

[0037]

[0038]

[0039] Among them, Center x and Center y are the horizontal and vertical coordinates of the center point respectively;

[0040] 3.2) According to step 3.1), the coordinates of the center point A (A) of the transmission tower body are obtained. x , A y ) and the center point coordinates of the top of the tower B(B x , B y ), calculate the tower inclination, including the following:

[0041] Calculate the connection vector between A and B, as shown in formula (8):

[0042]

[0043] Assuming the vertical direction is the positive direction, it is expressed as Then the vector and The angle cos(θ) is calculated as shown in equations (9) to (12):

[0044]

[0045]

[0046]

[0047]

[0048] When cos(θ) is equal to 1, it means that the angle θ is equal to 0, that is, the transmission tower is perpendicular to the ground; conversely, when cos(θ) is not equal to 1, it means that there is an angle θ that is not equal to 0, that is, the transmission tower is tilted.

[0049] The present invention uses remote sensing images and semantic segmentation technology to achieve rapid detection of the tilt of pole towers in a large area. Preprocessing of remote sensing images for shadow detection and noise cleaning, and using semantic segmentation technology in deep learning to replace the power grid inspection method can achieve efficient, accurate, and automated monitoring and evaluation of the tilt status of power transmission line pole towers, further enhancing the safety and stability of power grid operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 is a flow chart of the method of the present invention,

[0051] Figure 2 This is the DnCNN network structure diagram.

[0052] Figure 3 Schematic diagram of SE attention mechanism.

[0053] Figure 4 It is the structure diagram of U-Net neural network.

[0054] Figure 5 This is the CA module structure diagram.

[0055] Figure 6 It is the structure diagram of CNN neural network.

[0056] Figure 7 Schematic diagram of the transmission tower structure. DETAILED DESCRIPTION

[0057] like Figure 1 As shown, the present invention provides a method for detecting the inclination of power transmission line towers based on remote sensing image semantic segmentation, comprising the following steps:

[0058] Step 1): Preprocess the remote sensing image, including shadow detection and image denoising;

[0059] Step 2): The remote sensing image is semantically segmented through the improved U-Net neural network to extract the tower body and the top area of ​​the power transmission tower; attention and multi-scale detection are increased to meet the needs of complex scenes and small targets under remote sensing images;

[0060] Step 3): According to the semantic segmentation results of the transmission tower body and tower top, the coordinates of the center points of the two areas are calculated, and the inclination of the tower is calculated based on the coordinates of the two center points. By replacing manual inspections with semantic segmentation of remote sensing images, it is more efficient to detect whether the power tower is tilted.

[0061] Further, step 1) specifically includes:

[0062] 1.1) Considering that the texture and features in the shadow area of ​​the remote sensing image are partially blocked, it interferes with the subsequent detection of the tilt state of the power transmission line tower. Therefore, it is necessary to effectively and accurately detect the shadow area and remove it. 1 C 2 C 3 The concept of color space is to perform nonlinear transformation on the original RGB color space according to equations (1) to (3). In the equations, R, G, and B represent the brightness values ​​of the red, green, and blue channels of each pixel in the image, respectively.

[0063]

[0064]

[0065]

[0066] Due to C 3 The boundary between the shadow area and the non-shadow area in the channel is compared with C 1 C 2 More obvious, so based on C 3 The channel is used for shadow detection, and the variance filter function is defined to calculate the neighborhood variance of the pixel point, as shown in equations (4) and (5). In the equations, (i, j) represents the pixel position in the image, I represents the image pixel matrix, f(i, j) represents the pixel value, M represents the length of the image (unit pixel), and N represents the width of the image (unit pixel).

[0067]

[0068]

[0069] Since the pixel value jumps from the shadow area to the non-shadow area, the neighborhood variance of the boundary pixel point is higher, thus determining the shadow boundary.

[0070] 1.2) Considering that there are channel errors in the transmission process of remote sensing images, there are discrete isolated pixels on the image, that is, noise. Therefore, it is necessary to denoise the image before detecting the tilt state of the power transmission tower. The present invention improves the traditional DnCNN denoising neural network and adds the SE channel attention mechanism on the basis of the DnCNN network, such as Figure 2 As shown. The traditional DnCNN network consists of convolution Conv, activation function ReLU, and normalization BN. The SE module is added after the (Conv+BN+ReLU) layer. Before the SE module, the dimension of the feature map is H*W*C. The process of adding SE attention is as follows: Figure 3 As shown:

[0071] (1) First, perform spatial feature compression on the feature map and implement global average pooling in the spatial dimension to obtain a 1*1*C feature map;

[0072] (2) Through the fully connected layer FC learning, a feature map with channel attention is obtained, and its dimension is still 1*1*C;

[0073] The feature map of channel attention 1*1*C and the feature map of original input H*W*C are multiplied by the weight coefficient channel by channel, and finally the feature map with channel attention is output.

[0074] Further, step 2) specifically includes:

[0075] 2.1) Build a U-Net neural network. The U-Net neural network is a deep learning model designed for image segmentation tasks, and is well-known for its unique U-shaped network structure. U-Net consists of two parts: a downsampling part and an upsampling part. The downsampling part gradually extracts high-level features of the image through multiple convolution and pooling operations, while reducing the spatial dimension; the upsampling part gradually restores the spatial resolution of the image through upsampling (such as transposed convolution), and merges high- and low-level information by splicing with the feature maps of the corresponding layers in the contraction path. The network structure is as follows: Figure 4 Considering that the semantic segmentation effect of the traditional U-Net network on complex scenes and small objects is not good, the present invention improves the U-Net by adding an attention mechanism and multi-scale detection, which are steps 2.2) and 2.3) respectively.

[0076] 2.2) In order to improve the detection accuracy of the U-Net network in complex scenes of remote sensing images, the present invention adds a CA module that takes into account both spatial attention and channel attention in the downsampling part of the U-Net neural network, such as Figure 5 shown.

[0077] The workflow of the CA module is divided into two core stages: coordinate information embedding and coordinate attention generation. In the coordinate information embedding stage, the module first receives the feature map output by the previous convolutional layer as input. Subsequently, in order to capture important spatial information along the length (horizontal direction) and width (vertical direction) in the feature map, the module performs directional global pooling operations on the feature map respectively. The pooling results in these two directions are then concatenated along the channel dimension to form a tensor containing rich position information.

[0078] Entering the coordinate attention generation stage, the module first compresses and activates the spliced ​​coordinate information tensor to extract key feature representations. Then, through splitting and independent convolution operations, the module restores the original number of channels of the feature map and generates two independent attention weight maps for the length and width directions respectively. These two weight maps are then normalized by the Sigmoid function, and their output values ​​are limited to the interval (0,1) as the attention coefficients in their respective directions. These coefficients directly reflect the importance of different positions in the feature map.

[0079] Finally, in order to integrate the attention mechanism into the feature map, the module performs matrix multiplication on the original feature map and the attention coefficients in these two directions. This process actually performs spatial weighting on the feature map, emphasizing important areas while weakening the influence of secondary areas. In this way, the CA attention module effectively improves the model's ability to capture key features, thereby improving overall performance.

[0080] 2.3) In order to improve the detection capability of the U-Net network for small objects, the present invention adds a multi-scale feature extraction neural network before the U-Net downsampling part. Considering that the traditional CNN network is based on a single-scale feature map for detection, but a single-scale CNN is difficult to extract features effectively. Therefore, the present invention proposes a CNN neural network with multi-scale feature fusion, and the network structure is as follows: Figure 6 As shown in the figure, the model input first passes through convolutional layers with convolution kernels of (1×1, 3×3, 5×5), the results of different convolutional layers are concatenated together, and then the output is obtained through two fully connected layers.

[0081] Further, step 3) specifically includes:

[0082] 3.1) Based on the semantic segmentation results, the coordinates of the center point A of the tower body and the center point B of the tower top are calculated respectively. For the semantic segmentation results (region contour coordinates) of the two, the moments function of OpenCV is used to calculate the moments, including the zero-order moment (m 00 ) and first-order moment (m 10 ,m 01 ). The zero-order moment represents the total number of pixels within the contour, i.e., the area of ​​the contour. The first-order moment is related to the location of the centroid (or center point) of the contour. Specifically, m 10 is the weighted sum of the pixels of the contour along the x-axis, and m 01 is the weighted sum of the pixels of the contour along the y-axis, where “weighted” means that the contribution of each pixel varies according to its distance from the contour center. Based on the calculation results of the first-order moment, the coordinates of the center point can be calculated using formulas (6) and (7):

[0083]

[0084]

[0085] Among them, Center x and Center y are the horizontal and vertical coordinates of the center point, m 10 and m 01 is the first moment of the contour, m 00 is the zeroth moment of the contour.

[0086] 3.2) According to step 3.1), the coordinates of the center point A (A) of the transmission tower body are obtained. x , A y ) and the center point coordinates of the top of the tower B(B x , B y ), calculate the inclination of the tower, as shown in the schematic diagram Figure 7 As shown, the specific steps are as follows:

[0087] Calculate the connection vector between A and B, as shown in formula (8).

[0088]

[0089] Assuming the vertical direction is the positive direction, it is expressed as Then the vector and The angle cos(θ) is calculated as shown in equations (9) to (12).

[0090]

[0091]

[0092]

[0093]

[0094] When cos(θ) is equal to 1, it means that the angle θ is equal to 0, that is, the transmission tower is perpendicular to the ground. Conversely, when cos(θ) is not equal to 1, it means that there is an angle θ that is not equal to 0, that is, the transmission tower is tilted.

[0095] The present invention studies a method for detecting the inclination of power transmission line towers based on remote sensing image semantic segmentation, which has many important significances.

[0096] First, the use of remote sensing images and semantic segmentation technology can realize the rapid detection of the tilt of towers in a large area. Compared with traditional ground detection methods, such as manual inspection or the use of theodolites, total stations, etc., this method significantly improves detection efficiency and reduces manpower and time costs.

[0097] Secondly, through the high-precision semantic segmentation algorithm, the position of the tower in the remote sensing image can be accurately identified, and the key features of the tower, such as the center point coordinates of the tower body and the top of the tower, can be extracted. Based on these features, the tilt angle of the tower can be accurately calculated, which improves the accuracy of detection.

[0098] Then, the acquisition of remote sensing images is not restricted by geographical location and terrain conditions, and effective monitoring can be achieved even for remote areas or difficult-to-reach towers. This greatly reduces the difficulty and risk of on-site inspections.

[0099] Finally, the semantic segmentation algorithm based on deep learning has strong learning and adaptive capabilities, and can continuously optimize and improve the detection model. With the continuous advancement of technology and the continuous accumulation of data, the detection accuracy and efficiency of this method will be further improved.

[0100] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for detecting the tilt of power transmission line towers based on remote sensing image semantic segmentation, characterized in that: The following steps are involved: Step 1): Preprocess the remote sensing image, including shadow detection and image denoising; Step 2): Perform semantic segmentation on the remote sensing image through the improved U-Net neural network to extract the tower body and tower top area of ​​the power grid transmission tower; Step 3): According to the semantic segmentation results of the transmission tower body and tower top, the center point coordinates of the two areas are calculated, and the inclination of the tower is calculated based on the coordinates of the two center points.

2. The method for detecting the inclination of power transmission line towers based on remote sensing image semantic segmentation according to claim 1, characterized in that: Shadow detection in step 1) includes: The remote sensing image is introduced into the C1C2C3 color space, and the original RGB color space is nonlinearly transformed according to equations (1) to (3); In the formula, R, G, and B represent the brightness values ​​of the red, green, and blue channels of each pixel in the image, respectively; Based on the C3 channel, shadow detection is performed and the variance filter function is defined to calculate the neighborhood variance of the pixel point, as shown in equations (4) and (5); Where (i, j) represents the pixel position in the image, I represents the image pixel matrix, f(i, j) represents the pixel value, M represents the length of the image, and N represents the width of the image.

3. The method for detecting the inclination of power transmission line towers based on remote sensing image semantic segmentation according to claim 2, characterized in that: Image denoising in step 1) includes: The SE channel attention mechanism is added on the basis of the DnCNN network. Before the SE module, the dimension of the feature map is H*W*C. The process of adding SE attention is as follows: (1) First, perform spatial feature compression on the feature map and implement global average pooling in the spatial dimension to obtain a 1*1*C feature map; (2) Through the fully connected layer FC learning, a feature map with channel attention is obtained, and its dimension is still 1*1*C; the feature map of channel attention 1*1*C and the feature map of the original input H*W*C are multiplied by the weight coefficient channel by channel, and finally the feature map with channel attention is output.

4. The method for detecting the inclination of power transmission line towers based on remote sensing image semantic segmentation according to claim 1, characterized in that: Step 2) includes: 2.1) Build a U-Net neural network, including downsampling part and upsampling part; 2.2) Add a CA module that takes into account both spatial attention and channel attention in the downsampling part of the U-Net neural network; 2.3) Add a multi-scale feature extraction neural network before the U-Net downsampling part.

5. The method for detecting the inclination of power transmission line towers based on remote sensing image semantic segmentation according to claim 1, characterized in that: In step 2.2), The CA module includes: coordinate information embedding and coordinate attention generation; In the coordinate information embedding stage, the module first receives the feature map output by the previous convolutional layer as input; then, to capture the important spatial information along the length and width in the feature map, the module performs directional global pooling operations on the feature map respectively; the pooling results in these two directions are then concatenated along the channel dimension to form a tensor containing rich position information; Entering the coordinate attention generation stage, the module first compresses and activates the spliced ​​coordinate information tensor; then, through splitting and independent convolution operations, the module restores the original number of channels of the feature map and generates two independent attention weight maps for the length and width directions respectively; these two weight maps are then normalized by the Sigmoid function as the attention coefficients in their respective directions; Finally, the module performs matrix multiplication of the original feature map with the attention coefficients in these two directions.

6. The method for detecting the inclination of power transmission line towers based on remote sensing image semantic segmentation according to claim 1, characterized in that: In step 2.3), A CNN neural network with multi-scale feature fusion is adopted, and the network structure is as follows: the input first passes through convolution layers with convolution kernels of (1×1, 3×3, 5×5), the results of different convolution layers are spliced ​​together, and then the output is obtained through two fully connected layers.

7. The method for detecting the inclination of power transmission line towers based on remote sensing image semantic segmentation according to claim 1, characterized in that: Step 3) includes: 3.1) Based on the semantic segmentation results, calculate the coordinates of the center point A of the tower body and the center point B of the tower top respectively; Use OpenCV's moments function to calculate moments, including zero-order moments (m 00 ) and first-order moment (m 10 ,m 01 ); where the zero-order moment represents the total number of pixels within the contour, that is, the area of ​​the contour; the first-order moment m 10 is the weighted sum of the pixels of the contour along the x-axis, m 01 is the weighted sum of the pixels of the contour along the y-axis, and the center point coordinates are calculated using formulas (6) and (7): Among them, Center x and Center y are the horizontal and vertical coordinates of the center point respectively; 3.2) According to step 3.1), the coordinates of the center point A (A) of the transmission tower body are obtained. x , A y ) and the center point coordinates of the top of the tower B(B x , B y ), calculate the tower inclination, including the following: Calculate the connection vector between A and B, as shown in formula (8): Assuming the vertical direction is the positive direction, it is expressed as Then the vector and The angle cos(θ) is calculated as shown in equations (9) to (12): When cos(θ) is equal to 1, it means that the angle θ is equal to 0, that is, the transmission tower is perpendicular to the ground; conversely, when cos(θ) is not equal to 1, it means that there is an angle θ that is not equal to 0, that is, the transmission tower is tilted.

Images