An automated method for detecting the thickness of ice-covered transmission lines

Abstract

The present invention relates to the field of image processing technology, and more particularly to an automated ice-covered transmission line thickness detection method, comprising inputting an image of an ice-covered transmission line into an ice-covered transmission line thickness detection model, obtaining encoded features through an encoding path, fusing the encoded features with upsampled features, and inputting the features into a dual attention fusion module. The coordinate attention mechanism is first used to extract and concatenate the horizontal and vertical features of the input features, obtaining output features of the coordinate attention mechanism. The output features of the coordinate attention mechanism are then concatenated with the input features and input into a channel attention mechanism. Feature extraction is performed on each channel and its adjacent multiple adjacent channels, capturing local cross-channel information of the concatenated features. The segmentation result image of the ice-covered transmission line is output after a decoding path. The method reduces the interference of background information during the segmentation of ice-covered transmission lines and enhances the model's ability to identify information of ice-covered transmission lines of different scales, thereby improving the segmentation accuracy of the model.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an automatic method for detecting the thickness of ice-covered transmission lines. Background Art

[0002] Icing is a key concern for power transmission line maintenance during winter. In recent years, power grids across the country have experienced multiple rounds of icing each year, leading to numerous transmission line failures and even tower damage, posing a serious threat to the safe and stable operation of the power grid. Therefore, the development of a high-precision transmission line icing detection model is crucial for accident analysis and fault prediction in power grid maintenance.

[0003] Currently, the main methods for monitoring ice thickness on transmission lines include manual line inspection, image recognition, sag measurement, and embedded sensors. Image recognition has become a research hotspot for transmission line ice thickness monitoring due to its low initial investment, low operation and maintenance costs, high degree of automation, and accurate detection data. However, due to the interference of background information in on-site images of iced transmission lines, image recognition methods have difficulty accurately identifying iced transmission lines in images, making it difficult to quantitatively determine their ice thickness. Summary of the Invention

[0004] Therefore, the technical problem to be solved by the present invention is to overcome the problem that the image recognition method in the prior art is difficult to accurately identify the ice-covered transmission lines in the image, making it difficult to quantitatively judge the ice thickness thereof.

[0005] To solve the above technical problems, the present invention provides an automated ice-covered transmission line thickness detection method, comprising:

[0006] The image of ice-covered transmission lines is input into the ice-covered transmission line thickness detection model built based on the U-Net network, and the encoding features are obtained through the encoding path;

[0007] The encoded features and the upsampled features are spliced ​​together through jump connections to obtain a first fused feature, which is input into the dual attention fusion module. The horizontal features and vertical features of the first fused feature are first extracted and spliced ​​together through the coordinate attention mechanism to obtain the output features of the coordinate attention mechanism.

[0008] Splicing the output feature of the coordinate attention mechanism with the first fusion feature to obtain a spliced ​​feature;

[0009] The spliced ​​features are then input into the channel attention mechanism to perform feature extraction on each channel in the spliced ​​features and its multiple adjacent channels, capturing the local cross-channel information of the spliced ​​features and outputting the second fusion feature;

[0010] The second fused features are subjected to a decoding path to obtain decoded features, and a segmentation result map of the ice-covered transmission line is output;

[0011] The actual thickness of ice in the image is calculated based on the pixel area of ​​the transmission line in the segmentation result image of the ice-covered transmission line and the pixel area of ​​the transmission line in the reference image.

[0012] In one embodiment of the present invention, the horizontal features and vertical features of the first fusion feature are extracted and spliced ​​through the coordinate attention mechanism to obtain the output features of the coordinate attention mechanism, including:

[0013] Using horizontal global pooling and vertical global pooling to reduce the dimension of the first fusion feature, and splicing the horizontal global pooling feature with the vertical global pooling feature, and further extracting the spliced ​​features through two-dimensional convolution;

[0014] The extracted features are input into a normalization function for normalization processing, and then input into a nonlinear layer to capture different types of spatial information, and the number of channels of the feature map is restored to the same as the number of channels of the first fusion feature, thereby obtaining a horizontal feature map and a vertical feature map;

[0015] The horizontal feature map and the vertical feature map are input into the two-dimensional convolution and Sigmoid activation function respectively, and the output features of the coordinate attention mechanism are obtained after splicing.

[0016] In one embodiment of the present invention, the step of inputting the spliced ​​features into a channel attention mechanism, performing feature extraction on each channel in the spliced ​​features and its adjacent multiple neighboring channels, capturing local cross-channel information of the spliced ​​features, and outputting a second fused feature includes:

[0017] The concatenated features are reduced in dimension by global average pooling, and then each channel in the reduced-dimensional features and its adjacent multiple adjacent channels are extracted together to capture local cross-channel information, and then activated by a Sigmoid function;

[0018] The activated feature is superimposed on the splicing feature to obtain a second fusion feature.

[0019] In one embodiment of the present invention, the encoding path uses four encoding network modules to build a feature extraction network, wherein each encoding network module includes two 3×3 convolutional layers and one 2×2 maximum pooling layer.

[0020] In one embodiment of the present invention, the decoding path uses four decoding network modules to build a regression network, wherein each decoding network module includes two 3×3 convolutional layers and one 2×2 transposed convolutional layer.

[0021] In one embodiment of the present invention, the actual thickness of ice in the image is calculated based on the pixel area of ​​the transmission line in the segmentation result image of the ice-covered transmission line and the pixel area of ​​the transmission line in the reference image, using the formula:

[0022]

[0023] Where d represents the actual thickness of ice cover, d GT Indicates the diameter of the transmission line, S out represents the pixel area of ​​the ice-covered transmission line in the segmentation result map, S GT Represents the pixel area of ​​the power line in the reference image.

[0024] In one embodiment of the present invention, the training process of the ice-covered transmission line thickness detection model includes:

[0025] Using images of ice-covered power lines to train the ice-covered power line thickness detection model, and fine-tuning the model parameters according to a loss function;

[0026] Evaluation indicators are selected and the hyperparameters of the model are adjusted, and the ice-covered transmission line thickness detection model is repeatedly trained until the model network converges.

[0027] In one embodiment of the present invention, the image of the ice-covered power transmission line is annotated using a Segment Anything Model and inputted as training data into the ice-covered power transmission line thickness detection model.

[0028] In one embodiment of the present invention, before using images of ice-covered power lines to train the ice-covered power line thickness detection model, a general data set is used to pre-train the ice-covered power line thickness detection model.

[0029] In one embodiment of the present invention, the loss function is expressed as:

[0030]

[0031] Among them, C is the number of categories, y is the true label, which means each pixel belongs to a unique code of different categories. is the prediction result of the ice-covered transmission line thickness detection model, which represents the probability distribution of each pixel belonging to different categories.

[0032] The above technical solution of the present invention has the following advantages over the prior art:

[0033] The present invention provides an automated ice-covered transmission line thickness detection method, which constructs an ice-covered transmission line thickness detection model to segment ice-covered transmission line images, and adopts a dual attention fusion module to fuse coding features and upsampling features. The coordinate attention mechanism aggregating spatial features in the horizontal and vertical dimensions weakens the background information of the ice-covered transmission line image and emphasizes the features of the ice-covered transmission line, thereby filtering the background information and reducing the interference of the background information. The channel attention mechanism considers the mapping relationship between multiple channels without reducing the channel dimension, assigns different weights to each channel, and better identifies ice-covered transmission line information of different scales by fusing small-scale ice-covered transmission line information and large-scale semantic features, thereby promoting the model network to have both semantic features and detail features, and strengthening the model's ability to identify large- and small-scale ice-covered transmission line information, thereby improving the segmentation accuracy of the model.

[0034] The present invention provides an automated ice-covered transmission line thickness detection method. When training the ice-covered transmission line thickness detection model, SAM is used to automatically label unlabeled ice-covered transmission line data, saving manpower. A general data set is also used to pre-train the ice-covered transmission line thickness detection model, accelerating model convergence. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to make the content of the present invention more clearly understood, the present invention is further described in detail below based on specific embodiments of the present invention in conjunction with the accompanying drawings, wherein

[0036] Figure 1 This is a flow chart of the automated ice-covered transmission line thickness detection method provided by the present invention;

[0037] Figure 2 This is the overall network architecture diagram of the ice-covered transmission line thickness detection model provided by the present invention;

[0038] Figure 3 This is a network structure diagram of the dual attention fusion module used in the present invention;

[0039] Figure 4 This is a network structure diagram of the coordinate attention mechanism used in the present invention;

[0040] Figure 5 This is a network structure diagram of the channel attention mechanism used in the present invention;

[0041] Figure 6 is an ice-covered transmission line segmentation result diagram output by the ice-covered transmission line thickness detection model in an embodiment of the present invention;

[0042] Figure 7 2. ...

[0043] Figure 8This is a flow chart of an embodiment of the present invention for training an ice-covered transmission line thickness detection model. DETAILED DESCRIPTION

[0044] The present invention will be further described below with reference to the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it. However, the embodiments are not intended to limit the present invention.

[0045] Example 1

[0046] Reference Figure 1 As shown, the present invention provides an automated ice-covered transmission line thickness detection method, comprising:

[0047] Reference Figure 2 As shown, an image of an ice-covered power line is input into a U-Net-based ice-covered power line thickness detection model, where encoded features are obtained through an encoding path. The encoding path uses four encoding network modules to build a feature extraction network, where each encoding network module includes two 3×3 convolutional layers and one 2×2 max pooling layer to fully capture the deep semantic features in the image.

[0048] The encoding feature and the up-sampled feature are spliced ​​through jump connection to obtain the first fusion feature, which is input into the dual attention fusion module. The dual attention fusion module includes a coordinate attention mechanism and a channel attention mechanism. The structure is referenced. Figure 3 As shown, they are introduced in detail below.

[0049] Since images of ice-covered power lines in actual application scenarios often have complex background information, traditional networks find it difficult to eliminate the interference of background information, resulting in poor accuracy in the final segmentation results of ice-covered power lines. Therefore, it is necessary to reduce the interference caused by background information and highlight the ice-covered power lines. To this end, this paper introduces a coordinate attention mechanism (CA) to weaken background information to emphasize the characteristics of ice-covered power lines, guiding the model to focus on multi-scale ice-covered power lines information, thereby filtering background information and reducing the interference of background information. The specific operations are as follows:

[0050] After the first fusion feature is input into the dual attention fusion module, the horizontal feature and vertical feature of the first fusion feature are first extracted and spliced ​​through the coordinate attention mechanism to obtain the output feature of the coordinate attention mechanism. Figure 4 As shown, the process includes:

[0051] Using horizontal global pooling and vertical global pooling to reduce the dimension of the first fusion feature, respectively, to obtain two one-dimensional matrices, and then splicing the horizontal global pooling feature with the vertical global pooling feature, and further extracting the spliced ​​features through two-dimensional convolution;

[0052] The extracted features are input into a normalization function for normalization processing, and then input into a nonlinear layer to capture different types of spatial information, and the number of channels of the feature map is restored to the same as the number of channels of the first fusion feature, thereby obtaining a horizontal feature map and a vertical feature map;

[0053] The horizontal feature map and the vertical feature map are input into the two-dimensional convolution and Sigmoid activation function respectively, and the output features of the coordinate attention mechanism are obtained after splicing.

[0054] The coordinate attention mechanism can aggregate spatial features along the x and y dimensions, accurately and effectively highlighting the ice-covered power transmission line information in the image features, thereby better locating the ice-covered power transmission lines and improving the segmentation accuracy of the model.

[0055] The output feature of the coordinate attention mechanism is spliced ​​with the first fusion feature to obtain a spliced ​​feature.

[0056] In actual application scenarios, due to the principle of camera imaging, ice-covered power lines in the image appear larger when they are near and smaller when they are far away. This makes it difficult to extract rich information from small-scale ice-covered power lines at a distance and blur the image. To address the problem of different sizes of ice-covered power lines in ice-covered power line images, this paper introduces a channel attention mechanism (ECA). Figure 5 The specific operations are as follows:

[0057] The spliced ​​features are input into the channel attention mechanism, and the dimension is reduced by keeping the number of channels unchanged through global average pooling. Then, each channel in the reduced-dimensional features and its multiple adjacent channels are extracted together to properly capture local cross-channel information, and then activated by the Sigmoid function; the activated features are superimposed with the spliced ​​features to obtain the second fusion feature.

[0058] The channel attention mechanism considers the mapping relationship between multiple channels while maintaining channel dimensionality, assigning different weights to each channel and assigning a higher weight to the ice-covered power line information within the multi-scale features to emphasize this information. By fusing small-scale ice-covered power line information with large-scale semantic features, it better identifies ice-covered power lines at different scales, enabling the network to combine semantic and detailed features, strengthening its ability to identify both large- and small-scale ice-covered power line information, and thus improving the model's segmentation accuracy.

[0059] The second fusion feature is passed through the decoding path to obtain the decoded feature, and the segmentation result map of the ice-covered transmission line is output, as shown in Figure 6 shown.

[0060] The actual thickness of ice in the image is calculated based on the pixel area of ​​the ice-covered transmission line in the segmentation result image and the pixel area of ​​the ice-covered transmission line in the reference image. The output result is as follows: Figure 7 The formula for calculating the actual thickness of ice covering the transmission line is:

[0061]

[0062] Where d represents the actual thickness of ice cover, d GT Indicates the diameter of the transmission line, S out represents the pixel area of ​​the iced transmission line in the detection result map, S GT Represents the pixel area of ​​the power line in the reference image.

[0063] Example 2

[0064] Numerous inspection stations have accumulated a large number of monitoring images of ice-covered transmission lines during the long-term operation and maintenance of transmission lines. However, since professionals are required to accurately label the images and there are generally no benchmark values ​​for such data, such as the measured ice thickness and density, a large amount of high-quality image data is difficult to be effectively utilized, resulting in a lack of large-scale data sets that can be used for training.

[0065] Therefore, in order to realize automatic processing of unlabeled images, in this embodiment, a large model SAM is used to label the unlabeled data when training the ice-covered transmission line thickness detection model.

[0066] Reference Figure 8 As shown in Figure 2, the training process of the ice-covered transmission line thickness detection model includes:

[0067] Collect image data of ice-covered power lines and use the large model SAM (Segment Anything Model) to annotate the unlabeled data. Methods include: using label-studio + SAM for AI-assisted labeling, or directly using fine-tuned SAM to generate labeled mask images from unlabeled data.

[0068] The ice-covered transmission line thickness detection model is pre-trained using the general datasets VOC and COCO. The accuracy of the model on the test set during the pre-training process is recorded, and whether the pre-training process has converged is judged based on this. The weight of the network after convergence is recorded as the initial weight of the ice-covered transmission line thickness detection model network.

[0069] The pre-training process can save a lot of training time and computing resources, enable the ice-covered transmission line thickness detection model to learn common features, provide good initial weights, and provide a better starting point than random initialization in subsequent training, which helps to find the global optimal solution more quickly.

[0070] The pre-trained ice-covered transmission line thickness detection model is trained using mask images of labeled ice-covered transmission line images. The model parameters are fine-tuned according to the loss function, so that the model has better accuracy and feature extraction capabilities for specific application scenarios.

[0071] In this embodiment, the cross entropy function is used as the loss function, and the expression is:

[0072]

[0073] Among them, C is the number of categories, y is the true label, which means each pixel belongs to a unique code of different categories. is the prediction result of the ice-covered transmission line thickness detection model, which represents the probability distribution of each pixel belonging to different categories.

[0074] Evaluation indicators are selected and the hyperparameters of the model are adjusted, and the ice-covered transmission line thickness detection model is repeatedly trained until the model network converges.

[0075] The evaluation indicators selected in this embodiment are average precision (AP) and mean intersection over union (MIoU), which are expressed as follows:

[0076]

[0077]

[0078] Where N represents the number of images in a test sample vector, i represents the sample category, and AP i represents the accuracy of category i, that is, the area under the precision-recall curve of category i. The calculation formula of the accuracy is The calculation formula for recall is TP is the true positive, i.e., the number of samples correctly predicted by the model as category i, and FP is the false positive, i.e., the number of samples incorrectly predicted by the model as category i.

[0079] IoU i It represents the intersection-over-union ratio of category i, and the calculation formula is Inter refers to the intersection area, and Union refers to the union area.

[0080] In this embodiment, the training process uses the Adam optimizer, the initial learning rate is set to 0.0000001, and the momentum is set to 0.95.

[0081] The method for determining whether the model network has converged includes: recording the accuracy of the network on the validation set during training. When the accuracy fluctuation does not exceed 0.05 average precision (AP) in 10 rounds of training, the network can be considered to have converged.

[0082] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0083] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0084] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0085] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0086] Obviously, the above embodiments are merely examples for clarity of explanation and are not intended to limit the implementation methods. Those skilled in the art will appreciate that other variations or modifications can be made based on the above description. It is not necessary and impossible to enumerate all implementation methods here. Obvious variations or modifications arising therefrom remain within the scope of protection of the present invention.

Claims

1. An automated ice-covered transmission line thickness detection method, characterized in that: include: The image of ice-covered transmission lines is input into the ice-covered transmission line thickness detection model built based on the U-Net network, and the encoding features are obtained through the encoding path; The encoded features and the upsampled features are spliced ​​together through jump connections to obtain a first fused feature, which is input into the dual attention fusion module. The horizontal features and vertical features of the first fused feature are first extracted and spliced ​​together through the coordinate attention mechanism to obtain the output features of the coordinate attention mechanism. Splicing the output feature of the coordinate attention mechanism with the first fusion feature to obtain a spliced ​​feature; Then, the spliced ​​features are input into the channel attention mechanism, and features are extracted from each channel in the spliced ​​features and its adjacent multiple adjacent channels, and a second fusion feature is output; The second fused features are subjected to a decoding path to obtain decoded features, and a segmentation result map of the ice-covered transmission line is output; The actual thickness of ice in the image is calculated based on the pixel area of ​​the transmission line in the segmentation result image of the ice-covered transmission line and the pixel area of ​​the transmission line in the reference image.

2. The method for automatically detecting the thickness of ice-covered transmission lines according to claim 1, characterized in that: The horizontal features and vertical features of the first fusion feature are extracted and spliced ​​through the coordinate attention mechanism to obtain the output features of the coordinate attention mechanism, including: Using horizontal global pooling and vertical global pooling to reduce the dimension of the first fusion feature, and splicing the horizontal global pooling feature with the vertical global pooling feature, and further extracting the spliced ​​features through two-dimensional convolution; The extracted features are input into a normalization function for normalization processing, and then input into a nonlinear layer to capture different types of spatial information, and the number of channels of the feature map is restored to the same as the number of channels of the first fusion feature, thereby obtaining a horizontal feature map and a vertical feature map; The horizontal feature map and the vertical feature map are input into the two-dimensional convolution and Sigmoid activation function respectively, and the output features of the coordinate attention mechanism are obtained after splicing.

3. The method for automatically detecting the thickness of ice-covered transmission lines according to claim 1, characterized in that: Inputting the spliced ​​features into a channel attention mechanism, performing feature extraction on each channel in the spliced ​​features and its multiple adjacent channels, and outputting a second fusion feature, includes: The spliced ​​features are reduced in dimension by global average pooling, and then each channel in the reduced-dimensional features and its adjacent multiple adjacent channels are extracted together, and then activated by a Sigmoid function; the activated features are superimposed on the spliced ​​features to obtain a second fusion feature.

4. The method for automatically detecting the thickness of ice-covered transmission lines according to claim 1, characterized in that: The encoding path uses four encoding network modules to build a feature extraction network, where each encoding network module includes two 3×3 convolutional layers and one 2×2 maximum pooling layer.

5. The method for automatically detecting the thickness of ice-covered transmission lines according to claim 1, characterized in that: The decoding path uses four decoding network modules to build a regression network, where each decoding network module includes two 3×3 convolutional layers and one 2×2 transposed convolutional layer.

6. The method for automatically detecting the thickness of ice-covered transmission lines according to claim 1, characterized in that: The actual thickness of ice in the image is calculated based on the pixel area of ​​the transmission line in the segmentation result image of the ice-covered transmission line and the pixel area of ​​the transmission line in the reference image, using the formula: Where d represents the actual thickness of ice cover, d GT Indicates the diameter of the transmission line, S out represents the pixel area of ​​the ice-covered transmission line in the segmentation result map, S GT Represents the pixel area of ​​the power line in the reference image.

7. The method for automatically detecting the thickness of ice-covered transmission lines according to claim 1, characterized in that: The training process of the ice-covered transmission line thickness detection model includes: Using images of ice-covered power lines to train the ice-covered power line thickness detection model, and fine-tuning the model parameters according to a loss function; Evaluation indicators are selected and the hyperparameters of the model are adjusted, and the ice-covered transmission line thickness detection model is repeatedly trained until the model network converges.

8. The method for automatically detecting the thickness of ice-covered transmission lines according to claim 7, characterized in that: The image of the ice-covered transmission line is annotated using the Segment Anything Model and input into the ice-covered transmission line thickness detection model as training data.

9. The method for automatically detecting the thickness of ice-covered transmission lines according to claim 7, characterized in that: Before using the image of the ice-covered power line to train the ice-covered power line thickness detection model, the ice-covered power line thickness detection model is pre-trained using a general data set.

10. The method for automatically detecting the thickness of ice-covered transmission lines according to claim 7, characterized in that: The expression of the loss function is: Among them, C is the number of categories, y is the true label, which means each pixel belongs to a unique code of different categories. is the prediction result of the ice-covered transmission line thickness detection model, which represents the probability distribution of each pixel belonging to different categories.

Images