A power transmission line inspection method and device and a storage medium

Abstract

The application provides a power transmission line inspection method and device and a storage medium. The method comprises the following steps: determining a video image monitoring device arranged on the power transmission line; controlling the video image monitoring device to sequentially turn off a light supplement lamp, collect first original image data of the power transmission line, turn on the light supplement lamp, and collect second original image data of the power transmission line; extracting first target image data from the first original image data and second target image data from the second original image data; the first target image data and the second target image data are overlapped; determining icing on the power transmission line according to the first target image data; performing semantic segmentation of enhanced brightness contrast on the first target image data according to the second target image data to obtain local image data representing the power transmission line with icing; and classifying the icing type of the power transmission line according to the local image data. The embodiment effectively improves the accuracy of icing detection on the power transmission line.

Description

Technical Field

[0001] This invention belongs to the technical field of deep learning, and in particular relates to a method, equipment and storage medium for inspecting power transmission lines. Background Technology

[0002] During icy and snowy weather in winter, power transmission lines in the distribution network may be covered with ice (i.e., icing). To address this, the icing status of the transmission lines can be monitored, and de-icing measures can be taken in a timely manner to prevent icing from increasing the load on the transmission lines, reduce the occurrence of power outages and other faults, and ensure the safe operation of the distribution network.

[0003] Currently, one of the common methods for monitoring icing on power transmission lines is a purely visual approach, which involves collecting image data during power transmission line inspections and using algorithms such as semantic segmentation and classification to detect icing on the power transmission lines from the image data.

[0004] However, the shapes of icing on power transmission lines vary, and the shapes of some icing lines are quite similar to those of icing on natural landscapes (such as streams, patches of trees, mountain trails, etc.), which makes the accuracy of icing detection low. Summary of the Invention

[0005] In view of this, the present invention provides a method, equipment and storage medium for inspecting power transmission lines, so as to improve the accuracy of icing detection of power transmission lines.

[0006] A first aspect of the present invention provides a method for inspecting power transmission lines, comprising:

[0007] Determine the video surveillance devices to be installed on the transmission lines;

[0008] The video image monitoring device is controlled to sequentially turn off the fill light and collect the first raw image data of the power transmission line, and turn on the fill light and collect the second raw image data of the power transmission line.

[0009] First target image data is extracted from the first original image data, and second target image data is extracted from the second original image data; the first target image data and the second target image data overlap.

[0010] The transmission line is determined to be icy based on the first target image data;

[0011] For the ice-covered transmission line, semantic segmentation with enhanced brightness contrast is performed on the first target image data based on the second target image data to obtain local image data representing the ice-covered transmission line;

[0012] The icing type of the transmission line is classified based on the local image data.

[0013] A second aspect of the present invention provides an inspection device for power transmission lines, comprising:

[0014] The monitoring determination module is used to determine the video image monitoring devices installed on the transmission line;

[0015] The raw image data acquisition module is used to control the video image monitoring device to sequentially turn off the fill light and acquire the first raw image data of the power transmission line, and turn on the fill light and acquire the second raw image data of the power transmission line.

[0016] The target image data extraction module is used to extract first target image data from the first original image data and extract second target image data from the second original image data; the first target image data and the second target image data overlap.

[0017] An icing determination module is used to determine the icing of the transmission line based on the first target image data;

[0018] A semantic segmentation module is used to perform enhanced brightness contrast semantic segmentation on the first target image data based on the second target image data for the ice-covered transmission line, so as to obtain local image data representing the ice-covered transmission line.

[0019] An icing classification module is used to classify the icing type of the transmission line based on the local image data.

[0020] A third aspect of the present invention provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the transmission line inspection method as described in the first aspect above.

[0021] A fourth aspect of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the transmission line inspection method as described in the first aspect above.

[0022] The fifth aspect of the present invention provides a computer program product that, when run on a computer, causes the computer to perform the transmission line inspection method as described in the first aspect above.

[0023] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0024] In this embodiment, a video image monitoring device is installed on the transmission line; the video image monitoring device is controlled to sequentially turn off the supplementary light and collect first raw image data of the transmission line, and turn on the supplementary light and collect second raw image data of the transmission line; first target image data is extracted from the first raw image data, and second target image data is extracted from the second raw image data; the first target image data and the second target image data overlap; the transmission line is determined to be iced based on the first target image data; for the iced transmission line, semantic segmentation with enhanced brightness contrast is performed on the first target image data based on the second target image data to obtain local image data representing the iced transmission line; the icing type of the transmission line is classified based on the local image data. This embodiment constructs different environments by turning the supplementary light on and off. Under the condition that the transmission line is far from the ground, image data with obvious differences in brightness can be formed, which facilitates the identification of foreground and background, effectively improves the accuracy of semantic segmentation of iced transmission lines, eliminates background interference, and thus improves the accuracy of subsequent classification of iced transmission lines, thereby effectively improving the accuracy of icing detection of transmission lines. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a schematic diagram of a transmission line inspection method provided in an embodiment of the present invention;

[0027] Figure 2 This is a schematic diagram of the structure of a video image monitoring device provided in an embodiment of the present invention;

[0028] Figure 3 This is an example diagram of the first raw image data provided in an embodiment of the present invention;

[0029] Figure 4 This is an example diagram of the second original image data provided in an embodiment of the present invention;

[0030] Figure 5 This is a schematic diagram of the structure of an icing detection model provided in an embodiment of the present invention;

[0031] Figure 6 This is a schematic diagram of the structure of a first attention module provided in an embodiment of the present invention;

[0032] Figure 7This is a schematic diagram of the structure of a second attention module provided in an embodiment of the present invention;

[0033] Figure 8 This is a schematic diagram of a power transmission line inspection device provided in an embodiment of the present invention;

[0034] Figure 9 This is a schematic diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0035] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of the present invention. However, those skilled in the art will recognize that the present application may be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted to avoid unnecessary detail that could obscure the description of the present application.

[0036] The technical solution of the present invention will be illustrated below through specific embodiments.

[0037] Reference Figure 1 The diagram illustrates a method for inspecting power transmission lines according to an embodiment of the present invention, which may specifically include the following steps:

[0038] Step 101: Determine the video image monitoring device to be installed on the transmission line.

[0039] In this embodiment, multiple video image monitoring devices can be installed along the power transmission line.

[0040] For example, such as Figure 2 As shown, the pan-tilt unit in the video image monitoring device is a single-arm pan-tilt unit, which supports 360° horizontal rotation and ±90° vertical rotation. A fixed channel camera is installed at the base position. The pan-tilt camera compartment is equipped with a pan-tilt wide-angle camera, a pan-tilt zoom camera, and a rear-view camera, and supports wipers and fill lights.

[0041] The pan-tilt wide-angle camera has a resolution of ≥2 million pixels and a field of view of 100°, providing a wide monitoring perspective for large-scale monitoring of the environment surrounding power transmission lines.

[0042] The pan-tilt zoom camera has a resolution of ≥4 million pixels and a 40X optical zoom capability, enabling clear close-up shots of distant power transmission line equipment for easy observation of equipment details.

[0043] The rear-view camera has a resolution of ≥2 million pixels to supplement the monitoring perspective and ensure that the device can provide comprehensive coverage of all key areas of the power transmission line.

[0044] Edge nodes can generate inspection tasks for transmission lines periodically or irregularly. The inspection tasks include parameters such as the type of inspection (i.e., icing inspection), the inspection time, and the section to be inspected.

[0045] The edge node assigns inspection tasks to video image monitoring devices located in the inspected sections of the line. The video image monitoring devices then perform the inspection tasks, sequentially checking the transmission lines for icing.

[0046] Step 102: Control the video image monitoring device to sequentially turn off the supplementary light and collect the first raw image data of the transmission line, and turn on the supplementary light and collect the second raw image data of the transmission line.

[0047] The video image monitoring device is controlled to execute the first shooting operation and the second shooting operation in sequence. The first shooting operation can be executed first and then the second shooting operation, or the second shooting operation can be executed first and then the first shooting operation. This embodiment does not limit this.

[0048] For the first shooting operation, such as Figure 3 As shown, the video image monitoring device is controlled to turn off the supplementary light, so that the power transmission line is under natural light. At the same time, the camera is called to collect the first raw image data of the power transmission line.

[0049] For the second shooting operation, such as Figure 4 As shown, the video image monitoring device is controlled to turn on the supplementary light, so that the power transmission line is under a strong light source. At the same time, the camera is called to collect second raw image data of the power transmission line.

[0050] Generally, the distance between the power transmission line and the video image monitoring device is closer, while the distance between the ground and the video image monitoring device is farther. The distance between the power transmission line and the supplementary light of the video image monitoring device is closer, while the distance between the ground and the video image monitoring device is farther. The power transmission line exhibits a higher reflectivity under the supplementary light, and the reflectivity is even higher when covered with ice.

[0051] Therefore, power transmission lines exhibit higher reflectivity under supplementary lighting, and even higher reflectivity under icing conditions, resulting in higher brightness in the second original image data. In this way, without adding hardware (such as LiDAR), existing hardware (i.e., cameras and supplementary lighting) can be used to add high-contrast information (i.e., brightness) to the power transmission lines, facilitating the separation of foreground and background. The main engineering workload is focused on developing, testing, and deploying new detection algorithms, which are easy to promote.

[0052] The video image monitoring device can use mobile communication networks or Wi-Fi (wireless fidelity) to package and send the first and second original image data collected from each sampling point to the edge node.

[0053] Step 103: Extract the first target image data from the first original image data, and extract the second target image data from the second original image data.

[0054] Generally, when a video surveillance device is located at a high position, there is some shaking due to factors such as airflow disturbance (especially when the weather is cold). Therefore, the content of the first original image data is largely the same as that of the second original image data, but there are usually some differences. These differences are amplified by the distortion of the camera.

[0055] In edge nodes, for the same sampling point, the first original image data and the second original image data can be matched to extract the first target image data with the same part from the first original image data, and the second target image data with the same part from the second original image data, that is, the first target image data and the second target image data overlap.

[0056] In one embodiment of the present invention, step 103 may include the following steps:

[0057] Step 1031: Combine the brightness of pixels in the first original image data with the brightness of pixels in the previous frame's local image data to form a reflectivity threshold.

[0058] In this embodiment, the local image data of the transmission line (including the covered ice layer) in the previous frame can be retrieved. The brightness of the pixels in the first original image data and the brightness of the pixels in the local image data of the previous frame are fused into a reflectivity threshold in a linear or nonlinear manner. This smooths the real-time information and historical information to adapt to transmission line images with different light intensities and different degrees of ice reflectivity, improves the accuracy of the reflectivity threshold, and accurately locates the reflectivity area.

[0059] In practice, the power transmission lines to be inspected are mostly located in the field with complex backgrounds (such as mountains, trees, rivers, etc.). Therefore, a preset partitioning method can be used to divide the first original image data into multiple first image block data.

[0060] For each first image block data, the brightness of the pixels in the image block data is statistically calculated using the first average value and standard deviation. The multiple values ​​of the first average value and standard deviation are added together to obtain the lower brightness upper limit value.

[0061] At this point, the upper limit of low brightness can be expressed as: T1=μ+kσ, where T1 is the upper limit of low brightness, μ is the first average value, σ is the standard deviation, and k is the multiple, with k typically taking the value of [1.5, 2.5].

[0062] The second average value of the brightness of pixels in the previous frame of local image data is calculated.

[0063] For each first image block of data, the brightness threshold is obtained by adding the product of the lower limit value and the first weight, and the product of the second average value and the second weight; wherein the second weight is greater than the first weight.

[0064] At this point, the brightness threshold can be expressed as: T total =αT1+βT2, where T total T1 is the brightness threshold, T2 is the lower limit value, T2 is the second average value, α is the first weight, and β is the second weight.

[0065] Step 1032: If the brightness of a pixel in the second original image data is greater than or equal to the reflection threshold, then perform median filtering on the pixels in the second original image data to obtain candidate image data.

[0066] The brightness of the pixels in the second original image data is compared with the reflection threshold. If the brightness of the pixels in the second original image data is greater than or equal to the reflection threshold, it means that the pixels in the second original image data may belong to the power transmission line (including the ice layer covering it). At this time, median filtering (such as a window of 5×5) can be performed on the pixels in the second original image data to obtain candidate image data, preserve the surrounding texture, and avoid the appearance of solid color blocks.

[0067] Furthermore, if the first original image data is divided into multiple first image block data, the second original image data can be divided into multiple second image block data using the same partitioning method.

[0068] For the second image block data with the same location as the first image block data, if the brightness of the pixels in the second image block data is greater than or equal to the reflection threshold, then median filtering is performed on the pixels in the second image block data to obtain candidate image data.

[0069] In complex contexts, the brightness baselines of different areas vary greatly (e.g., rivers are bright, mountains are dark), and segmented private reflective threshold traversal can improve the accuracy of locating local reflective areas.

[0070] Step 1033: Align the first original image data with the candidate image data to obtain the overlapping region.

[0071] In this embodiment, operators such as SIFT (Scale Invariant Feature Transform) can be used to calculate feature points from the first original image data and the candidate image data, respectively.

[0072] Feature points that overlap with pixels whose brightness is greater than or equal to the reflection threshold are removed from the second original image data to avoid incorrect matching caused by reflection.

[0073] The feature points of the first original image data are matched with the feature points of the candidate image data. Algorithms such as RANSAC (Random Sample Consensus) are used to remove feature points that are not matched. The feature points that are matched successfully are used to solve the accurate transformation matrix. This can handle translation, rotation, scaling and even slight perspective distortion caused by the shaking of video image monitoring devices.

[0074] The first original image data and the candidate image data are aligned using a transformation matrix. The overlapping area after alignment of the first original image data and the candidate image data is found by the intersection of the non-black region masks.

[0075] Step 1034: Extract the first target image data located in the overlapping region from the first original image data, and extract the second target image data located in the overlapping region from the second original image data.

[0076] In this embodiment, overlapping areas are cropped from the first original image data and the second original image data. First target image data located in the overlapping area is extracted from the first original image data, and second target image data located in the overlapping area is extracted from the second original image data. This eliminates the non-overlapping parts caused by image misalignment and can be directly used for subsequent icing semantic segmentation.

[0077] Step 104: Determine the icing of the transmission line based on the first target image data.

[0078] In this embodiment, image analysis can be performed on the first target image data to determine whether the transmission line is covered with ice.

[0079] In practical implementation, an icing detection model can be determined, such as... Figure 5 As shown, the icing detection model includes a first main encoder, a second main encoder, a first head structure, and a second head structure.

[0080] The first master encoder and the second master encoder both include structures such as convolutional layers, pooling layers (such as max pooling layers), and activation functions (such as ReLU). The first head structure and the second head structure both include structures such as convolutional layers, fully connected layers, and activation functions.

[0081] The first target image data is input into the first main encoder to extract shallow features, which are then used as the first main image features.

[0082] The first master image features are input into the second master encoder to extract high-level features, which are then used as the second master image features.

[0083] On the one hand, the second main image features are input into the first head structure to extract features and classify them to identify the first probability of unice-covered strips.

[0084] On the other hand, the second main image features are input into the second head structure to extract features and classify them to identify the second probability of ice strips.

[0085] Among them, the strip-shaped objects mainly include power transmission lines.

[0086] If the first probability is less than or equal to the first icing threshold and the second probability is greater than or equal to the second icing threshold, then the transmission line is determined to be iced.

[0087] If the first probability is greater than the first icing threshold and / or the second probability is less than the second icing threshold, then the transmission line is determined to be uniced.

[0088] In this embodiment, the first main image feature and the second main image feature are encoded features for semantic segmentation. Reusing the first main image feature and the second main image feature for cross-classification not only reduces development costs, but also improves the accuracy of detecting ice accretion in complex backgrounds, avoids invalid semantic segmentation, and reduces the amount of subsequent computation.

[0089] Step 105: For the icy transmission line, perform semantic segmentation with enhanced brightness contrast on the first target image data based on the second target image data to obtain local image data representing the icy transmission line.

[0090] If the transmission line is covered with ice, semantic segmentation can be performed by combining the first target image data and the second target image data. During the semantic segmentation process, the difference in image brightness under natural light and supplementary lighting is used for comparison to enhance the semantic segmentation. Pixels representing the icy transmission line are segmented from the first target image data to obtain the current local image data.

[0091] In one embodiment of the present invention, step 105 may include the following steps:

[0092] Step 1051: Input the second target image data into the first encoder to extract the first image features that enhance brightness contrast.

[0093] In this embodiment, as Figure 5As shown, the icing detection model includes a first encoder. The second target image data is input into the first encoder for encoding. The encoding is enhanced by comparing the different brightness of the images under natural light and supplementary lighting to obtain the first image features.

[0094] For example, the first encoder includes a first encoding block and a first attention module.

[0095] The structure of the first coding block is the same as that of the first main encoder, and the two are twin structures. In this case, the encoding of the second target image data adds a brightness enhancement compared to the encoding of the first target image data.

[0096] In this example, the second target image data is input into the first coding block for encoding, thereby extracting the first level of image features.

[0097] like Figure 6 As shown, the following operations are performed in the first attention module:

[0098] S11. Compress the channels in the first-level image features to obtain the first reference image features.

[0099] The channels in the first-level image features are compressed to the original channel 1 / C (C is a constant, such as 2, 4, etc.), thereby focusing on the core features and reducing the amount of computation.

[0100] Furthermore, the first-level image features can be input into the convolutional layer Conv to extract features. Batch normalization (BN) and ReLU (Rectified Linear Unit) activation operations are performed on the features output by the convolutional layer Conv to compress the channels in the first-level image features and obtain the first reference image features.

[0101] Therefore, the process of compressing the channel can be represented as: A1=ReLU(BN(Conv(X))), where X is the first-level image feature and A1 is the first reference image feature.

[0102] S12. Extract the brightness features of the local space from the features of the first reference image and use them as the features of the second reference image.

[0103] In general, the ice layer covering power transmission lines is a continuous space. Therefore, the brightness features of the local space can be extracted from the features of the first reference image and used as the features of the second reference image.

[0104] Furthermore, the features of the first reference image can be input into the convolutional layer Conv to extract features, and batch normalization (BN) and ReLU activation operations can be performed on the features output by the convolutional layer Conv to extract the local spatial brightness features from the first reference image features, which are then used as the second reference image features.

[0105] Therefore, the process of extracting the brightness features of the local space can be expressed as: A2=ReLU(BN(Conv(A1))), where A1 is the first reference image feature and A2 is the second reference image feature.

[0106] S13. Recover the channels of the second reference image features to obtain the third reference image features.

[0107] The features of the second reference image are input into the convolutional layer Conv to perform a convolution operation, thereby restoring the channels of the second reference image features and obtaining the features of the third reference image, ensuring dimension matching.

[0108] Therefore, the process of restoring the channel can be expressed as: A3=Conv(A2), where A2 is the second reference image feature and A3 is the third reference image feature.

[0109] S14. Activate the features of the third reference image to obtain the first attention weights for brightness.

[0110] In this embodiment, the Sigmoid function can be used to activate the features of the third reference image (mapped to the range [0, 1]) to obtain the first attention weight W1 for brightness.

[0111] S15. Multiply the first-level image features element-wise with the first attention weights to obtain the first secondary image features that enhance brightness contrast.

[0112] In this embodiment, the first-level image feature X is multiplied element-wise with the first attention weight W1 (Hadamard product). That is, each element in the first-level image feature is multiplied with the first attention weight at the corresponding position to obtain the first secondary image feature T that enhances brightness contrast, thereby amplifying the bright area features and suppressing the dark area features.

[0113] Step 1052: Input the first image feature into the second image encoder to extract the second image feature that enhances brightness contrast.

[0114] In this embodiment, as Figure 5 As shown, the icing detection model includes a second encoder. The first image data is input into the second encoder for encoding. The encoding is enhanced by comparing the different brightness of the images under natural light and supplementary lighting, thereby obtaining the second image features.

[0115] For example, the second encoder includes a second encoding block and a second attention module.

[0116] The structure of the second coding block is the same as that of the second master encoder, and the two are twin structures. In this case, the encoding of the first slave image features is enhanced with an additional brightness enhancement compared to the encoding of the first master image features.

[0117] In this example, the first image feature is input into the second coding block for encoding, thereby extracting the second level of image features.

[0118] like Figure 7 As shown, the following operations are performed in the second attention module:

[0119] S21. Compress the channels in the second-level image features to obtain the fourth reference image features.

[0120] The channels in the second-level image features are compressed to the original channel 1 / C (C is a constant, such as 2, 4, etc.), thereby focusing on the core features and reducing the amount of computation.

[0121] Furthermore, the second-level image features can be input into the convolutional layer Conv to extract features, and batch normalization (BN) and ReLU activation operations can be performed on the features output by the convolutional layer Conv to compress the channels in the second-level image features and obtain the fourth reference image features.

[0122] Therefore, the process of compressing the channel can be represented as: A4=ReLU(BN(Conv(Y))), where Y is the second-level image feature and A4 is the fourth reference image feature.

[0123] S22. Extract the local spatial brightness features from the features of the fourth reference image and use them as the features of the fifth reference image.

[0124] In general, the ice layer covering the transmission line is a continuous space. Therefore, the brightness features of the local space can be extracted from the features of the fourth reference image and used as the features of the fifth reference image.

[0125] Furthermore, the features of the fourth reference image can be input into the convolutional layer Conv to extract features, and batch normalization (BN) and ReLU activation operations can be performed on the features output by the convolutional layer Conv to extract the local spatial brightness features from the features of the fourth reference image as the features of the fifth reference image.

[0126] Therefore, the process of extracting the brightness features of the local space can be expressed as: A5=ReLU(BN(Conv(A4))), where A4 is the fourth reference image feature and A5 is the fifth reference image feature.

[0127] S23. Recover the channels of the fifth reference image features to obtain the sixth reference image features.

[0128] The features of the fifth reference image are input into the convolutional layer Conv to perform a convolution operation, thereby restoring the channels of the features of the fifth reference image and obtaining the features of the sixth reference image, ensuring dimension matching.

[0129] Therefore, the process of restoring the channel can be represented as: A6=Conv(A5), where A5 is the fifth reference image feature and A6 is the sixth reference image feature.

[0130] S24. Activate the features of the sixth reference image to obtain the second attention weights for brightness.

[0131] In this embodiment, the sixth reference image features can be activated using the Sigmoid function (mapped to the range [0, 1]) to obtain the second attention weight W2 for brightness.

[0132] S25. Multiply the second-level image features element-wise with the second attention weights to obtain the second image features that enhance brightness contrast.

[0133] In this embodiment, the second-level image feature Y is multiplied element-wise by the second attention weight W2 (Hadamard product). That is, each element in the second-level image feature is multiplied by the second attention weight at the corresponding position to obtain the second secondary image feature T2 that enhances brightness contrast, thereby amplifying the bright area features and suppressing the dark area features.

[0134] Step 1053: Input the second master image features into the bridging layer to generate the first reconstructed image features.

[0135] In this embodiment, the icing detection model includes a bridging layer, which includes structures such as convolutional layers and activation functions (e.g., ReLU). The second main image features are input into the bridging layer for processing (e.g., dimensionality compression, feature flattening, etc.) to generate the first reconstructed image features.

[0136] Step 1054: Input the first reconstructed image features and the second image features into the first decoder and decode them into the second reconstructed image features.

[0137] In this embodiment, the icing detection model includes a first decoder, which includes structures such as an upsampling layer, a convolutional layer, and an activation function (such as ReLU).

[0138] The first reconstructed image feature and the second derived image feature are concatenated to form the first comprehensive image feature. The first comprehensive image feature is then input into the first decoder for decoding to obtain the second reconstructed image feature.

[0139] Step 1055: Input the second reconstructed image features and the first image features into the second decoder to decode into semantic reconstructed image data.

[0140] In this embodiment, the icing detection model includes a second decoder, which includes structures such as an upsampling layer, a convolutional layer, and an activation function (such as ReLU).

[0141] The reconstructed image features are concatenated with the first derived image features to form a second comprehensive image feature. The second comprehensive image feature is then input into the first decoder for decoding to obtain semantically reconstructed image data.

[0142] In this semantically reconstructed image data, each pixel has semantic information, which includes the background and local image data representing icy power transmission lines.

[0143] In this embodiment, the second target image data and the first target image data are essentially the same. The main difference between the two is the brightness contrast of the transmission line (including the covered ice layer). The high-level features of the second target image data replace the high-level features of the first target image data for residual connection (skip connection). That is, the shallow features of the first target image data and the high-level features of the second target image data are combined for decoding, so that the brightness contrast is used as the basis for segmentation and localization, thereby improving the accuracy of semantic segmentation of the transmission line (including the covered ice layer).

[0144] Step 106: Classify the icing type of the transmission line based on local image data.

[0145] In this embodiment, a classification operation can be performed on local image data to obtain the type of icing on the transmission line, such as rime, hoarfrost, mixed rime, etc., thereby assessing the harm caused by the ice covering the transmission line to the transmission line and formulating corresponding de-icing measures to remove the ice covering the transmission line.

[0146] In practice, the morphology of ice layers of the same type formed under different climatic conditions can vary. Therefore, an ice feature library can be constructed, which contains sample ice image features under different ice types and records the climatic conditions under which the sample ice image features were formed.

[0147] For the current local image data, the icing image features of the samples under each icing type can be determined from the icing feature library. Then, machine learning or deep learning methods can be used to extract features from the local image data as the target icing image features.

[0148] Calculate the similarity between the features of the target icing image and the features of each sample icing image.

[0149] If the similarity between the transmission line and a certain sample icing image feature is greater than or equal to the icing threshold, the transmission line is determined to be the icing type corresponding to the sample icing image feature, and the target icing image feature is stored in the icing feature library as a new sample icing image feature under that icing type.

[0150] Compared to directly using machine learning or deep learning for classification, accumulating the features of sample icing images can not only adapt to the changing morphology of icing under complex climate conditions, but also provide a data foundation for subsequent longer-term icing management by mining data such as similarity and climate conditions (e.g., studying changing trends, clustering, etc.).

[0151] In this embodiment, a video image monitoring device is installed on the transmission line; the video image monitoring device is controlled to sequentially turn off the supplementary light and collect first raw image data of the transmission line, and turn on the supplementary light and collect second raw image data of the transmission line; first target image data is extracted from the first raw image data, and second target image data is extracted from the second raw image data; the first target image data and the second target image data overlap; the transmission line is determined to be iced based on the first target image data; for the iced transmission line, semantic segmentation with enhanced brightness contrast is performed on the first target image data based on the second target image data to obtain local image data representing the iced transmission line; the icing type of the transmission line is classified based on the local image data. This embodiment constructs different environments by turning the supplementary light on and off. Under the condition that the transmission line is far from the ground, image data with obvious differences in brightness can be formed, which facilitates the identification of foreground and background, effectively improves the accuracy of semantic segmentation of iced transmission lines, eliminates background interference, and thus improves the accuracy of subsequent classification of iced transmission lines, thereby effectively improving the accuracy of icing detection of transmission lines.

[0152] It should be noted that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0153] Reference Figure 8 The diagram shows a schematic of a transmission line inspection device provided in an embodiment of the present invention, which may specifically include the following modules:

[0154] The monitoring determination module 801 is used to determine the video image monitoring devices installed on the transmission line.

[0155] The raw image data acquisition module 802 is used to control the video image monitoring device to sequentially turn off the fill light and acquire the first raw image data of the power transmission line, and turn on the fill light and acquire the second raw image data of the power transmission line.

[0156] The target image data extraction module 803 is used to extract first target image data from the first original image data and extract second target image data from the second original image data; the first target image data and the second target image data overlap.

[0157] Ice accretion determination module 804 is used to determine the ice accretion of the transmission line based on the first target image data;

[0158] The semantic segmentation module 805 is used to perform semantic segmentation with enhanced brightness contrast on the first target image data based on the second target image data for the ice-covered transmission line, so as to obtain local image data representing the ice-covered transmission line.

[0159] The icing classification module 806 is used to classify the icing type of the transmission line based on the local image data.

[0160] In one embodiment of the present invention, the target image data extraction module 803 is further configured to:

[0161] The brightness of pixels in the first original image data is combined with the brightness of pixels in the previous frame's local image data to form a reflectance threshold;

[0162] If the brightness of a pixel in the second original image data is greater than or equal to the reflectivity threshold, then median filtering is performed on the pixels in the second original image data to obtain candidate image data.

[0163] Align the first original image data with the candidate image data to obtain the overlapping region;

[0164] Extract first target image data located in the overlapping region from the first original image data, and extract second target image data located in the overlapping region from the second original image data.

[0165] In one embodiment of the present invention, the target image data extraction module 803 is further configured to:

[0166] The first original image data is divided into multiple first image block data;

[0167] For each of the first image block data, the brightness of the pixels in the image block data is statistically calculated using a first average value and a standard deviation. The first average value and a multiple of the standard deviation are then added together to obtain the lower brightness upper limit value.

[0168] The second average brightness of pixels in the previous frame of local image data is calculated.

[0169] For each of the first image block data, the product of the lower limit value and the first weight, and the product of the second average value and the second weight are added together to obtain the brightness threshold; wherein, the second weight is greater than the first weight.

[0170] If the brightness of a pixel in the second original image data is greater than or equal to the reflectivity threshold, then median filtering is performed on the pixels in the second original image data to obtain candidate image data, including:

[0171] The second original image data is divided into multiple second image block data;

[0172] For the second image block data with the same position as the first image block data, if the brightness of the pixels in the second image block data is greater than or equal to the reflection threshold, then median filtering is performed on the pixels in the second image block data to obtain candidate image data.

[0173] In one embodiment of the present invention, the icing determination module 804 is further configured to:

[0174] The first target image data is input into the first main encoder to extract the first main image features;

[0175] The first main image features are input into the second main encoder to extract the second main image features;

[0176] The second main image features are input into the first head structure to identify the first probability of uncovered ice strips;

[0177] The second main image features are input into the second head structure to identify the second probability of ice-covered strips;

[0178] If the first probability is less than or equal to the first icing threshold and the second probability is greater than or equal to the second icing threshold, then the transmission line is determined to be iced.

[0179] In one embodiment of the present invention, the semantic segmentation module 805 is further configured to:

[0180] The second target image data is input into the first encoder to extract first image features that enhance brightness contrast.

[0181] The first image feature is input into the second image encoder to extract a second image feature that enhances brightness contrast.

[0182] The second main image features are input into the bridging layer to generate the first reconstructed image features;

[0183] The first reconstructed image feature and the second derived image feature are input into the first decoder and decoded into the second reconstructed image feature;

[0184] The second reconstructed image features and the first image features are input into the second decoder to decode into semantic reconstructed image data; the semantic reconstructed image data includes local image data representing the icing of the transmission line.

[0185] In one embodiment of the present invention, the first slave encoder includes a first encoding block and a first attention module; the structure of the first encoding block is the same as the structure of the first master encoder;

[0186] The semantic segmentation module 805 is also used for:

[0187] The second target image data is input into the first coding block to extract the first level of image features;

[0188] In the first attention module, the channels in the first-level image features are compressed to obtain the first reference image features. The brightness features of the local space are extracted from the first reference image features as the second reference image features. The channels of the second reference image features are recovered to obtain the third reference image features. The third reference image features are activated to obtain the first attention weight with respect to brightness. The first-level image features are multiplied element-wise with the first attention weight to obtain the first secondary image features that enhance brightness contrast.

[0189] In one embodiment of the present invention, the second slave encoder includes a second encoding block and a second attention module; the structure of the second encoding block is the same as the structure of the second master encoder;

[0190] The semantic segmentation module 805 is also used for:

[0191] The first image feature is input into the second coding block to extract the second-level image feature;

[0192] In the second attention module, the channels in the second-level image features are compressed to obtain the fourth reference image features. The brightness features of the local space are extracted from the fourth reference image features as the fifth reference image features. The channels of the fifth reference image features are restored to obtain the sixth reference image features. The sixth reference image features are activated to obtain the second attention weights related to brightness. The second-level image features are multiplied element-wise with the second attention weights to obtain the second secondary image features that enhance brightness contrast.

[0193] In one embodiment of the present invention, the icing classification module 806 is further configured to:

[0194] Determine the icing image features of samples under each icing type;

[0195] Extract target icing image features from the local image data;

[0196] Calculate the similarity between the features of the target icing image and the features of the sample icing image;

[0197] If the similarity is greater than or equal to the icing threshold, then the transmission line is determined to be the icing type corresponding to the icing image features of the sample.

[0198] The present invention provides a transmission line inspection device, which can realize the steps in the aforementioned transmission line inspection method embodiments.

[0199] It should be noted that the module division in the various transmission line inspection devices provided in the above embodiments is illustrative and only represents a logical functional division. In actual implementation, other division methods may also be used. Furthermore, the functional modules in the various embodiments of this invention can be integrated into a single processor, exist as separate physical entities, or be integrated into a single module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0200] If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the technical solution of the embodiments of the present invention can be embodied in the form of a computer program product, which is stored in a computer storage medium and includes several instructions to cause an electronic device or processor to execute all or part of the steps of the methods in the various embodiments of the present invention. The aforementioned computer storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0201] Furthermore, the transmission line inspection device and transmission line inspection method provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0202] Reference Figure 9 The diagram illustrates an electronic device according to an embodiment of the present invention. Figure 9As shown, the electronic device in this embodiment of the invention includes: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps in the above-described transmission line inspection method embodiment. Alternatively, when the processor executes the computer program, it implements the functions of each module in the above-described transmission line inspection device embodiment.

[0203] For example, the computer program may be divided into one or more modules, which are stored in the memory and executed by the processor to complete this application. The one or more modules may be a series of computer program instruction segments capable of performing a specific function, which can be used to describe the execution process of the computer program in the electronic device.

[0204] The electronic device may be a desktop computer, a cloud server, or other computing device. The electronic device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that... Figure 9 This is merely one example of an electronic device and does not constitute a limitation on the electronic device. It may include more or fewer components than illustrated, or combine certain components, or different components. For example, the electronic device may also include input / output devices, network access devices, buses, etc.

[0205] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0206] The memory can be an internal storage unit of the electronic device, such as a hard drive or RAM. Alternatively, it can be an external storage device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, etc. Furthermore, the memory can include both internal and external storage units. The memory is used to store the computer program and other programs and data required by the electronic device. The memory can also be used to temporarily store data that has been output or will be output.

[0207] This invention also discloses an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the transmission line inspection method as described in the foregoing embodiments.

[0208] This invention also discloses a computer-readable storage medium storing a computer program that, when executed by a processor, implements the transmission line inspection method as described in the foregoing embodiments.

[0209] This invention also discloses a computer program product that, when run on a computer, causes the computer to execute the transmission line inspection method described in the foregoing embodiments.

[0210] The embodiments described above are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for inspecting power transmission lines, characterized in that, include: Determine the video surveillance devices to be installed on the transmission lines; The video image monitoring device is controlled to sequentially turn off the fill light and collect the first raw image data of the power transmission line, and turn on the fill light and collect the second raw image data of the power transmission line. Extract first target image data from the first original image data, and extract second target image data from the second original image data; The first target image data overlaps with the second target image data; The transmission line is determined to be icy based on the first target image data; For the ice-covered transmission line, semantic segmentation with enhanced brightness contrast is performed on the first target image data based on the second target image data to obtain local image data representing the ice-covered transmission line; The icing type of the transmission line is classified based on the local image data; The step of extracting first target image data from the first original image data and extracting second target image data from the second original image data includes: The brightness of pixels in the first original image data is combined with the brightness of pixels in the previous frame's local image data to form a reflectance threshold; If the brightness of a pixel in the second original image data is greater than or equal to the reflectivity threshold, then median filtering is performed on the pixels in the second original image data to obtain candidate image data. Align the first original image data with the candidate image data to obtain the overlapping region; Extract first target image data located in the overlapping region from the first original image data, and extract second target image data located in the overlapping region from the second original image data; The step of fusing the brightness of pixels in the first original image data with the brightness of pixels in the previous frame's local image data to form a reflectivity threshold includes: The first original image data is divided into multiple first image block data; For each of the first image block data, the brightness of the pixels in the image block data is statistically calculated using a first average value and a standard deviation. The first average value and a multiple of the standard deviation are then added together to obtain the lower brightness upper limit value. The second average brightness of pixels in the previous frame of local image data is calculated. For each of the first image block data, the product of the lower limit value and the first weight, and the product of the second average value and the second weight are added together to obtain the brightness threshold; wherein, the second weight is greater than the first weight. If the brightness of a pixel in the second original image data is greater than or equal to the reflectivity threshold, then median filtering is performed on the pixels in the second original image data to obtain candidate image data, including: The second original image data is divided into multiple second image block data; For the second image block data with the same position as the first image block data, if the brightness of the pixels in the second image block data is greater than or equal to the reflection threshold, then median filtering is performed on the pixels in the second image block data to obtain candidate image data.

2. The method for inspecting transmission lines according to claim 1, characterized in that, Determining the icing of the transmission line based on the first target image data includes: The first target image data is input into the first main encoder to extract the first main image features; The first main image features are input into the second main encoder to extract the second main image features; The second main image features are input into the first head structure to identify the first probability of uncovered ice strips; The second main image features are input into the second head structure to identify the second probability of ice-covered strips; If the first probability is less than or equal to the first icing threshold and the second probability is greater than or equal to the second icing threshold, then the transmission line is determined to be iced.

3. The method for inspecting transmission lines according to claim 2, characterized in that, The step of performing semantic segmentation with enhanced brightness contrast on the first target image data based on the second target image data to obtain local image data representing the icing transmission line includes: The second target image data is input into the first encoder to extract first image features that enhance brightness contrast. The first image feature is input into the second image encoder to extract a second image feature that enhances brightness contrast. The second main image features are input into the bridging layer to generate the first reconstructed image features; The first reconstructed image feature and the second derived image feature are input into the first decoder and decoded into the second reconstructed image feature; The second reconstructed image features and the first image features are input into the second decoder to decode into semantic reconstructed image data; the semantic reconstructed image data includes local image data representing the icing of the transmission line.

4. The method for inspecting transmission lines according to claim 3, characterized in that, The first encoder includes a first encoding block and a first attention module; The structure of the first coding block is the same as that of the first main encoder; The step of inputting the second target image data into the first encoder to extract the first image features that enhance brightness contrast includes: The second target image data is input into the first coding block to extract the first level of image features; In the first attention module, the channels in the first-level image features are compressed to obtain the first reference image features. The brightness features of the local space are extracted from the first reference image features as the second reference image features. The channels of the second reference image features are recovered to obtain the third reference image features. The third reference image features are activated to obtain the first attention weight with respect to brightness. The first-level image features are multiplied element-wise with the first attention weight to obtain the first secondary image features that enhance brightness contrast.

5. The method for inspecting transmission lines according to claim 3, characterized in that, The second encoder includes a second encoding block and a second attention module; The structure of the second coding block is the same as that of the second main encoder; The step of inputting the first image feature into the second image encoder to extract the second image feature that enhances brightness contrast includes: The first image feature is input into the second coding block to extract the second-level image feature; In the second attention module, the channels in the second-level image features are compressed to obtain the fourth reference image features. The brightness features of the local space are extracted from the fourth reference image features as the fifth reference image features. The channels of the fifth reference image features are restored to obtain the sixth reference image features. The sixth reference image features are activated to obtain the second attention weights related to brightness. The second-level image features are multiplied element-wise with the second attention weights to obtain the second secondary image features that enhance brightness contrast.

6. The method for inspecting transmission lines according to any one of claims 1, 3-5, characterized in that, The classification of icing types of the transmission lines based on the local image data includes: Determine the icing image features of samples under each icing type; Extract target icing image features from the local image data; Calculate the similarity between the features of the target icing image and the features of the sample icing image; If the similarity is greater than or equal to the icing threshold, then the transmission line is determined to be the icing type corresponding to the icing image features of the sample.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the transmission line inspection method as described in any one of claims 1-6.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the transmission line inspection method as described in any one of claims 1-6.

Images