Power transmission line icing detection method, device and equipment based on deep learning

Abstract

The invention provides a power transmission line icing detection method, device and equipment based on deep learning. The method comprises the following steps: obtaining a power transmission line diagram with a suspected icing area; an icing initial feature extraction model is called to process the power transmission line diagram, multiple first multi-scale icing initial feature maps are obtained, and the multiple first multi-scale icing initial feature maps are composed of intermediate results and final results output by the model; an icing deep feature extraction model is called to process the multiple first multi-scale icing initial feature maps, multiple second multi-scale icing deep feature maps are obtained, and the multiple second multi-scale icing deep feature maps are composed of intermediate results and final results output by the model; and calling an icing position determination model, performing prediction processing on the plurality of second multi-scale icing deep feature maps by using a plurality of prediction heads in the icing position determination model, and determining position prediction information of an icing area in the power transmission line map.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method, apparatus, and equipment for detecting icing on power transmission lines based on deep learning. Background Technology

[0002] Icing on power transmission lines is a significant threat to the safe and stable operation of the power grid. Traditional de-icing methods typically rely on manual inspections or fixed threshold activation of de-icing devices, which suffer from problems such as slow response, inaccurate positioning, and high energy consumption. In recent years, deep learning technology has made significant progress in areas such as image recognition and anomaly detection, providing new ideas for accurate icing detection.

[0003] However, even with the use of deep learning, existing methods for de-icing transmission lines still have the following problems in the process of detecting icing locations:

[0004] (1) Existing detection models have poor robustness in harsh natural environments. Especially under extreme weather conditions such as fog and haze and heavy snowfall, the difference in optical characteristics between the icing area and the surrounding environment is reduced, and the image quality is severely affected by atmospheric scattering and noise, which significantly reduces the accuracy of the model in judging the icing boundary and thickness changes.

[0005] (2) Existing detection methods still have shortcomings for small-scale icing areas or icing areas obscured by structures. Traditional convolutional neural networks are not fine enough in their feature extraction layer design, making it difficult to capture the detailed features of small icing areas. Furthermore, when the target part is obscured, it is easy to miss recognition due to incomplete feature information.

[0006] (3) The icing area is visually similar to other parts of the transmission line, making it difficult for the model to distinguish them accurately. Most current methods rely mainly on single visual features and fail to make full use of scene context information for comprehensive discrimination, thus affecting detection accuracy. Summary of the Invention

[0007] To address the aforementioned technical problems, embodiments of the present invention provide a deep learning-based method for detecting icing on transmission lines, comprising:

[0008] Obtain a map of the power transmission lines with suspected icing areas;

[0009] The transmission line diagram is processed by calling the initial icing feature extraction model to obtain multiple first multi-scale initial icing feature maps. The multiple first multi-scale initial icing feature maps involve different feature scales and are all within the first scale range. The multiple first multi-scale initial icing feature maps are composed of intermediate results and the final result output by the model.

[0010] The ice-deep feature extraction model is called to process multiple first multi-scale ice-deep feature maps to obtain multiple second multi-scale ice-deep feature maps. The multiple second multi-scale ice-deep feature maps involve different feature scales and are all within the second scale range. The multiple second multi-scale ice-deep feature maps are composed of intermediate results and the final result output by the model.

[0011] The icing location determination model is invoked, and multiple prediction heads in the icing location determination model are used to perform prediction processing on multiple second multi-scale icing depth feature maps to determine the location prediction information of the icing area in the transmission line map.

[0012] In one embodiment, constructing the initial feature extraction model for icing includes:

[0013] An initial model for processing two-dimensional image data is constructed. The initial model includes multiple convolutional processing modules and a multi-dimensional feature extraction module. The convolutional processing module is formed by at least one convolutional module, a batch normalization module, and an activation function module. The multi-dimensional feature extraction module includes three processing links. Each processing link includes a dilated convolution sub-module, a normalization sub-module, and an activation function sub-module. The activation function sub-module is used to introduce a smooth nonlinear mechanism in the negative range. The first processing link and the second processing link both include a dilation rate parameter, and the dilation rate parameters are different.

[0014] The initial model is trained to obtain the initial feature extraction model for icing.

[0015] In one embodiment, when processing the input multi-scale feature map based on the multi-dimensional feature extraction module, the process includes:

[0016] The input multi-scale feature map is processed through the first processing link to obtain the first multi-dimensional icing feature map;

[0017] The input multi-scale feature map is processed through the second processing link to obtain a second multi-dimensional icing feature map;

[0018] The first multi-dimensional icing feature map and the second multi-dimensional icing feature map are fused in the channel dimension to obtain the third multi-dimensional icing feature map.

[0019] The input multi-scale feature map is processed through the third processing link to obtain the fourth multi-dimensional icing feature map;

[0020] The third multidimensional icing feature map is multiplied element-wise with the multi-scale feature map to obtain the fifth multidimensional icing feature map.

[0021] The fifth multi-dimensional icing feature map is multiplied element-wise with the fourth multi-dimensional icing feature map to obtain the first multi-scale icing initial feature map for output.

[0022] In one embodiment, the convolutional processing module and the multi-dimensional feature extraction module are set at intervals. The first multi-scale icing initial feature map generated by the convolutional processing module or the multi-dimensional feature extraction module of the previous layer is input into the convolutional processing module or the multi-dimensional feature extraction module of the next layer for further processing. The first multi-scale icing initial feature map generated by the multi-dimensional feature extraction module is simultaneously sent to the deep icing feature extraction model for processing.

[0023] In one embodiment, a deep feature extraction model for ice embankment is constructed, comprising:

[0024] Multiple adaptive icing location attention submodules, pooling submodules, upsampling submodules, feature fusion submodules, and convolution submodules are constructed. The adaptive icing location attention submodule is used to simultaneously process the obtained multi-scale icing feature map through at least three processing lines to perform feature extraction processing at different degrees. The multi-scale icing feature map includes a first multi-scale initial icing feature map or a second multi-scale deep icing feature map passed from the previous feature extraction unit.

[0025] Multiple sub-modules are connected sequentially in a preset order to form the ice-covered deep feature extraction model with a multi-layer structure;

[0026] The feature fusion submodule is used to fuse the second multi-scale icing depth feature map passed from the previous layer and the first multi-scale icing depth feature map generated by the initial icing feature extraction model on the channel, and to pass the generated fused multi-scale icing depth feature map to the next layer.

[0027] In one embodiment, processing the multi-scale icing feature map obtained based on the adaptive icing location attention submodule includes:

[0028] Each processing line performs a first convolution operation and a second convolution operation on the same multi-scale icing feature map to obtain a first adaptive icing position attention map for each processing line. The first convolution operation is a dilated convolution operation, and the second convolution operation is an adaptive variable stride convolution operation.

[0029] Multiple first adaptive icing location attention maps are fused based on the channel dimension, and horizontal pooling and vertical pooling operations are performed to obtain horizontal and vertical output results.

[0030] The horizontal and vertical output results are normalized respectively.

[0031] The normalized horizontal output and vertical output are fused based on the channel dimension to obtain the second adaptive icing position attention map;

[0032] The second adaptive icing location attention map is multiplied element-wise with the initially obtained multi-scale icing feature map to obtain the third adaptive icing location attention map that forms the second multi-scale icing depth feature map.

[0033] In one embodiment, constructing the icing location determination model includes:

[0034] Construct at least three prediction heads and a filtering module connected to the at least three prediction heads to form the icing location determination model based on the at least three prediction heads and the filtering module;

[0035] Specifically, at least three prediction heads are used to obtain second multi-scale icing depth feature maps with different scales output by the icing depth feature extraction model at different stages, and to perform regression convolution and classification convolution operations on the obtained second multi-scale icing depth feature maps to obtain sub-predicted icing location feature maps. The filtering module is used to calculate and filter the location prediction information of the icing area in the transmission line diagram based on all the sub-predicted icing location feature maps.

[0036] In one embodiment, the icing location determination model is invoked to predict multiple second multi-scale icing depth feature maps, including:

[0037] Each prediction head is used to predict the area where the transmission line is located based on the obtained second multi-scale icing depth feature map, and the confidence level of the presence of icing targets in each prediction area is calculated, and the confidence level is associated with the corresponding prediction area.

[0038] The filtering module merges the sub-predicted icing location feature maps marked with the predicted region and confidence level output by each prediction head, and uses a non-maximum suppression algorithm and a preset cross-union ratio threshold to calculate and filter the predicted region and confidence level, thereby determining the target predicted region with the highest confidence level and a degree of mutual overlap lower than the cross-union ratio threshold.

[0039] The filtering module is used to mark the target prediction region on the merged feature map.

[0040] Another embodiment of the present invention also provides a deep learning-based transmission line icing detection device, comprising:

[0041] The first acquisition module is used to obtain a diagram of a power transmission line with suspected icing areas;

[0042] The first calling module is used to call the initial icing feature extraction model to process the transmission line diagram and obtain multiple first multi-scale initial icing feature maps. The multiple first multi-scale initial icing feature maps involve different feature scales and are all within the first scale range. The multiple first multi-scale initial icing feature maps are composed of intermediate results and the final result output by the model.

[0043] The second calling module is used to call the ice-covered deep feature extraction model to process multiple first multi-scale ice-covered initial feature maps to obtain multiple second multi-scale ice-covered deep feature maps. The multiple second multi-scale ice-covered deep feature maps involve different feature scales and are all within the second scale range. The multiple second multi-scale ice-covered deep feature maps are composed of intermediate results and the final results output by the model.

[0044] The third calling module is used to call the icing location determination model, and use multiple prediction heads in the icing location determination model to perform prediction processing on multiple second multi-scale icing depth feature maps respectively, so as to determine the location prediction information of the icing area in the transmission line map.

[0045] Another embodiment of the present invention also provides an electronic device, comprising:

[0046] One or more processors;

[0047] Memory, configured to store one or more programs;

[0048] When the one or more programs are executed by the one or more processors, the one or more processors implement the deep learning-based transmission line icing detection method as described in any one of the above descriptions.

[0049] Based on the above, the beneficial effects of the embodiments of this application include that the proposed deep learning-based transmission line icing detection method can efficiently, accurately, and automatically identify the icing situation on transmission lines, providing precise guidance for subsequent icing-melting operations of the icing-melting device.

[0050] Other features and advantages of this application will be set forth in the following description and will be apparent in part from the description or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings.

[0051] The technical solution of this application will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0052] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0053] Figure 1 This is a flowchart illustrating the deep learning-based transmission line icing detection method in an embodiment of the present invention.

[0054] Figure 2 This is a schematic diagram of the main structure of the power transmission line icing detection network in an embodiment of the present invention.

[0055] Figure 3 This is a schematic diagram of the application process of the MLIFE module in an embodiment of the present invention.

[0056] Figure 4 This is a schematic diagram of the application process of the ATLAM module in an embodiment of the present invention.

[0057] Figure 5 This is a structural block diagram of a deep learning-based transmission line icing detection device according to an embodiment of the present invention. Detailed Implementation

[0058] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings, but these are not intended to limit the scope of the invention.

[0059] It should be understood that various modifications can be made to the embodiments disclosed herein. Therefore, the following description should not be considered as limiting, but merely as an example of embodiments. Other modifications within the scope of this disclosure will be apparent to those skilled in the art.

[0060] The accompanying drawings, which are included in and form part of this specification, illustrate embodiments of the present disclosure and, together with the general description of the disclosure given above and the detailed description of the embodiments given below, serve to explain the principles of the disclosure.

[0061] These and other features of the invention will become apparent from the following description of preferred forms of embodiments given as non-limiting examples, with reference to the accompanying drawings.

[0062] It should also be understood that although the invention has been described with reference to some specific examples, those skilled in the art can certainly implement many other equivalent forms of the invention, which have the features described in the claims and are therefore all within the scope of protection defined herein.

[0063] The above and other aspects, features and advantages of this disclosure will become more apparent when taken in conjunction with the accompanying drawings and in view of the following detailed description.

[0064] Specific embodiments of the present disclosure are described thereafter with reference to the accompanying drawings; however, it should be understood that the disclosed embodiments are merely examples of the present disclosure and can be implemented in various ways. Well-known and / or repeated functions and structures are not described in detail to avoid unnecessary or redundant details that could obscure the present disclosure. Therefore, the specific structural and functional details disclosed herein are not intended to be limiting, but merely to serve as the basis and representative basis for the claims to teach those skilled in the art to use the present disclosure in a variety of substantially any suitable detailed structures.

[0065] This specification may use the phrases “in one embodiment,” “in another embodiment,” “in yet another embodiment,” or “in still another embodiment,” all of which may refer to one or more of the same or different embodiments according to this disclosure.

[0066] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0067] like Figure 1 As shown, this embodiment of the invention provides a deep learning-based method for detecting icing on power transmission lines, comprising:

[0068] S1: Obtain a diagram of power transmission lines with suspected icing areas;

[0069] S2: Call the initial icing feature extraction model to process the transmission line diagram to obtain multiple first multi-scale initial icing feature maps. The multiple first multi-scale initial icing feature maps involve different feature scales and are all within the first scale range. The multiple first multi-scale initial icing feature maps are composed of intermediate results and the final result output by the model.

[0070] S3: Call the ice-covered deep feature extraction model to process multiple first multi-scale ice-covered initial feature maps to obtain multiple second multi-scale ice-covered deep feature maps. The multiple second multi-scale ice-covered deep feature maps involve different feature scales and are all within the second scale range. The multiple second multi-scale ice-covered deep feature maps are composed of intermediate results and the final results output by the model.

[0071] S4: Call the icing location determination model, and use multiple prediction heads in the icing location determination model to perform prediction processing on multiple second multi-scale icing depth feature maps respectively, to determine the location prediction information of the icing area in the transmission line map.

[0072] Based on the above, this embodiment pre-constructs multiple models with different functions to perform hierarchical, progressive feature recognition on the obtained transmission line map with suspected icing areas. This achieves both shallow and deep feature extraction; features from clear areas in the image are extracted, as are features from less clear and visually obscure areas. Simultaneously, the extracted shallow features are used by the icing deep feature extraction model, fusing the shallow and deep feature maps for the next iteration of deep feature extraction. This ensures the extracted deep features are more comprehensive and accurate, accurately reflecting the actual icing state and enabling subsequent icing location determination models to predict the actual icing location based on the extracted features. For example, the de-icing device can directly obtain the predicted location information to pinpoint the location requiring de-icing.

[0073] Based on the above, it can be seen that the deep learning-based transmission line icing detection method proposed in this embodiment can achieve efficient, accurate, and automated identification of icing conditions on transmission lines. The overall judgment and prediction errors are small, effectively solving the shortcomings of current methods that rely on manual observation of images to determine whether there is icing and the large errors in machine identification of icing. The icing location information predicted by the method in this embodiment can provide accurate guidance for subsequent de-icing operations, improve de-icing efficiency, and reduce the difficulty of de-icing.

[0074] In one embodiment, imaging equipment (including but not limited to drones, fixed-position high-definition cameras, etc.) can be used to photograph the area where the transmission line is located, acquiring images of the actual state of the transmission line. Then, the transmission line images are labeled with information about icing conditions. This involves using labeling tools (such as, but not limited to, Label Img, Labelme, VOTT, CVAT, SuperAnnotate, DataLoop, and RectLabel) to select suspected icing locations on the transmission line, determining the coordinates of the top-left and bottom-right vertices of the box, and marking the suspected icing locations as bounding boxes. Each bounding box is assigned a classification label to indicate the icing condition of the transmission line within that box. These classification labels are of two types: one indicating icing on the transmission line, and the other indicating no icing on the transmission line. Through labeling, the system can filter the transmission line images, obtaining images labeled with suspected icing, and defining these images as transmission line maps with suspected icing areas.

[0075] In this embodiment, the initial ice-covered feature extraction model, the deep ice-covered feature extraction model, and the ice-covered location determination model all need to be pre-built and trained before they can be used normally.

[0076] In this embodiment, the initial feature extraction model for ice accumulation is constructed, including:

[0077] S5: Construct an initial model for two-dimensional image data processing. The initial model includes multiple convolution processing modules and a multi-dimensional feature extraction module. The convolution processing module is formed by at least one convolution module, a batch normalization module, and an activation function module. The multi-dimensional feature extraction module includes three processing links. Each processing link includes a dilated convolution sub-module, a normalization sub-module, and an activation function sub-module. The activation function sub-module is used to introduce a smooth nonlinear mechanism in the negative range. The first processing link and the second processing link both include a dilation rate parameter, and the dilation rate parameters are different.

[0078] S6: Train the initial model to obtain the initial feature extraction model for ice accretion.

[0079] When processing the input multi-scale feature map based on the multi-dimensional feature extraction module, the following steps are included:

[0080] S7: The input multi-scale feature map is processed through the first processing link to obtain the first multi-dimensional icing feature map;

[0081] S8: The input multi-scale feature map is processed through the second processing link to obtain a second multi-dimensional icing feature map;

[0082] S9: The first multi-dimensional icing feature map and the second multi-dimensional icing feature map are fused in the channel dimension to obtain the third multi-dimensional icing feature map;

[0083] S10: The input multi-scale feature map is processed through the third processing link to obtain the fourth multi-dimensional icing feature map;

[0084] S11: Multiply the third multi-dimensional icing feature map element-wise with the multi-scale feature map to obtain the fifth multi-dimensional icing feature map;

[0085] S12: Multiply the fifth multi-dimensional icing feature map and the fourth multi-dimensional icing feature map element by element to obtain the first multi-scale icing initial feature map for output.

[0086] In this embodiment, the convolution processing module and the multi-dimensional feature extraction module are set at intervals. The first multi-scale icing initial feature map generated by the convolution processing module or the multi-dimensional feature extraction module of the previous layer is input into the convolution processing module or the multi-dimensional feature extraction module of the next layer for further processing. The first multi-scale icing initial feature map generated by the multi-dimensional feature extraction module is simultaneously sent to the icing deep feature extraction model for processing.

[0087] Specifically, in combination Figure 2 As shown, the convolution module in this embodiment is a Conv_BN_ReLU module, but the specific module is not limited. In this embodiment, the Conv_BN_ReLU module is first applied to F0 for convolution, batch normalization, and activation function operations to obtain the initial multi-scale icing feature map F1; then, the next layer's Conv_BN_ReLU module is applied to F1 for convolution, batch normalization, and activation function operations to obtain the initial multi-scale icing feature map F2; subsequently, the first MLIFE module (multi-dimensional feature extraction module) of this layer is applied to F2 to obtain the initial multi-scale icing feature map F3; then, the next layer's Conv_BN_ReLU module is applied to F3 for convolution, batch normalization, and activation function operations to obtain the initial multi-scale icing feature map F4, and... F4 is input into the second MLIFE module for processing to obtain the initial multi-scale icing feature map F5. Subsequently, the next layer's Conv_BN_ReLU module is applied to F5 for convolution, batch normalization, and activation function operations to obtain the initial multi-scale icing feature map F6. F6 is then input into the third MLIFE module for processing to obtain the initial multi-scale icing feature map F7. Finally, the last layer's Conv_BN_ReLU module is applied to F7 for convolution, batch normalization, and activation function operations to obtain the initial multi-scale icing feature map F8. F8 is then input into the fourth MLIFE module for processing to obtain the initial multi-scale icing feature map F9.

[0088] In this embodiment, a total of four convolutional processing modules and a multi-dimensional feature extraction module are set. Each layer includes a convolutional processing module and a multi-dimensional feature extraction module. In actual applications, more layers can be set, or the number of layers can be reduced. The specific configuration is not fixed and can be flexibly configured according to actual needs.

[0089] The multi-dimensional feature extraction module proposed in this embodiment is specifically named the Multi-dimensional Line Icing Feature Extraction sub-module, or MLIFE module for short. The overall structure diagram of the MLIFE module can be found in [reference needed]. Figure 3 As shown. The MLIFE module in this embodiment has three processing links, as follows: Figure 3 As shown, they are labeled a1, a2, and a3, respectively. Among them, a1 is the shallow-scale pooling feature extraction route; Lb is the mid-scale pooling feature extraction route; and Lc is the deep-scale pooling feature extraction route.

[0090] When applying, continue to combine Figure 2 As shown, the initial feature map F2 of the multi-scale icing is used as the input feature map of the MLIFE module. For ease of explanation, F2 is denoted as Y0 here.

[0091] In processing link a1, firstly, Y0 is convolved with a dilated convolution of size 3×3 and a dilation rate of 5 to obtain a multi-dimensional line icing feature map Y1. Then, Y1 is normalized by BN and then fed into the HSELU activation function to obtain a multi-dimensional line icing feature map Y2.

[0092] In this embodiment, an HSELU activation function is designed in the MLIFE module. The formula for the HSELU activation function is as follows:

[0093]

[0094] In the formula, x is the original input value processed by the activation function, which comes from the output of the previous layer in the neural network; n and m are two random parameters that can change with the calculation, and n+m=1.

[0095] In deep learning, neuron "death" typically refers to the phenomenon where certain neurons (especially those using ReLU-like activation functions) permanently stop updating during training. Specifically, when the weighted input of a neuron remains consistently negative (e.g., after weight updates or changes in input data distribution), the ReLU output becomes fixed at 0 (i.e., "dead"), and the gradient also becomes 0, preventing the neuron from being activated or having its parameters adjusted in subsequent training. This is usually caused by a high learning rate or improper weight initialization, reducing the model's expressive power and training effectiveness. The HSELU activation function proposed in this embodiment introduces smooth non-linearity within the negative range, which helps prevent neuron "death" and enhances spatial dependencies between pixels, enabling the capture of pixel-level visual information.

[0096] Furthermore, in processing link a2, firstly, Y0 is convolved using a dilated convolution with an inflation rate of 7 and a size of 3×3 to obtain a multi-dimensional line icing feature map Y3. Then, Y3 is normalized by BN and fed into the HSELU activation function for activation to obtain a multi-dimensional line icing feature map Y4.

[0097] In processing link a3, firstly, Y0 is convolved using a 3×3 ordinary convolution, then normalized by BN, and finally fed into the HSELU activation function for activation to obtain the multi-dimensional line icing feature map Y5.

[0098] Simultaneously, after obtaining Y2 and Y4, the module will fuse Y2 and Y4 in the channel dimension (i.e. Figure 3 The concat operation (in the model) is performed, and after adjusting the number of channels using a 1×1 ordinary convolution, a multi-dimensional line icing feature map Y6 is obtained. Then, Y0 and Y6 are multiplied element-wise (i.e., ...). Figure 3The multiplication operation in the process yields the multi-dimensional line icing feature map Y7. Then, Y7 and Y5 are multiplied element-wise to obtain the multi-dimensional line icing feature map Y8. This multi-dimensional line icing feature map Y8 is the feature map output by the MLIFE module.

[0099] In another embodiment, a feature extraction model for deep ice layers is constructed, including:

[0100] S13: Construct multiple adaptive icing location attention submodules, pooling submodules, upsampling submodules, feature fusion submodules, and convolution submodules; the adaptive icing location attention submodule is used to simultaneously process the obtained multi-scale icing feature map through at least three processing lines to perform feature extraction processing at different degrees, the multi-scale icing feature map includes the first multi-scale initial icing feature map or the second multi-scale deep icing feature map passed from the previous feature extraction unit;

[0101] S14: Connect multiple sub-modules in a preset order to form the ice-covered deep feature extraction model with a multi-layer structure;

[0102] The feature fusion submodule is used to fuse the second multi-scale icing depth feature map passed from the previous layer and the first multi-scale icing depth feature map generated by the initial icing feature extraction model on the channel, and to pass the generated fused multi-scale icing depth feature map to the next layer.

[0103] Furthermore, when processing the multi-scale icing feature map obtained based on the adaptive icing location attention submodule, the following steps are included:

[0104] S15: Perform a first convolution operation and a second convolution operation on the same multi-scale icing feature map obtained by each of the processing lines to obtain a first adaptive icing position attention map corresponding to each of the processing lines. The first convolution operation is a dilated convolution operation, and the second convolution operation is an adaptive variable stride convolution operation.

[0105] S16: Fuse multiple first adaptive icing position attention maps based on the channel dimension, and perform horizontal pooling and vertical pooling operations to obtain horizontal output results and vertical output results;

[0106] S17: Normalize the horizontal and vertical output results respectively;

[0107] S18: The normalized horizontal output result and the vertical output result are fused based on the channel dimension to obtain the second adaptive icing position attention map;

[0108] S19: Multiply the second adaptive icing location attention map element-wise with the initially obtained multi-scale icing feature map to obtain the third adaptive icing location attention map that forms the second multi-scale icing depth feature map.

[0109] For example, continue to combine Figure 2 As shown, the structure of the ice-covered deep feature extraction model in this embodiment is as follows: Figure 2 As shown, when applying the icing depth feature extraction model for image processing, the first ATLAM module is used to process F9 to obtain the adaptive line icing depth feature map F10; then SPPF (pooling submodule, used to fuse global information at different scales through a fast spatial pyramid pooling method) is used to process F10 to obtain the adaptive line icing depth feature map F11, and then an upsampling operation (i.e., ...) is performed. Figure 2 The Upsample operation (executed by the Upsample submodule) yields the adaptive line icing depth feature map F12. Then, the Fusion submodule fuses F7 (transmitted by the MLIFE module in the shallow icing feature extraction model) with F12 along the channel dimension to obtain the adaptive line icing depth feature map F13. Subsequently, the second ATLAM module processes F13 to obtain the adaptive line icing depth feature map F14, and then upsamples F14 to obtain the adaptive line icing depth feature map F15. Finally, the Fusion submodule fuses F5 and F15 (transmitted by the MLIFE module in the shallow icing feature extraction model) along the channel dimension to obtain the adaptive line icing depth feature map F16.

[0110] Next, F16 is input into the third ATLAM module to obtain the adaptive line icing depth feature map F17; the added convolutional submodule is then applied, such as... Figure 2 The Conv_BN_ReLU module performs convolution, batch normalization, and activation function operations on F17 to obtain the adaptive deep feature map of line icing, F18. Then, the fusion submodule fuses F18 with F14 along the channel dimension (i.e., ...). Figure 3 The adaptive line icing depth feature map F19 is obtained by performing a Concat operation (in the first step). Then, the fourth ATLAM module is applied to process F19 to obtain the adaptive line icing depth feature map F20. The next Conv_BN_ReLU module is then applied to F20 to perform convolution, batch normalization, and activation function operations to obtain the adaptive line icing depth feature map F21. Subsequently, F11 and F21 are fused along the channel dimension to obtain the adaptive line icing depth feature map F22. Finally, the fifth ATLAM module is applied to process F23 to obtain the adaptive line icing depth feature map F23. In this embodiment, the adaptive line icing depth feature map is the adaptive icing position attention map.

[0111] Furthermore, the adaptive icing position attention sub-module in this embodiment is specifically named the Adaptive Transmission Line ice-coating Attention sub-module, or ATLAM module for short.

[0112] The overall structure of the ATLAM module is as follows: Figure 4 As shown in the figure. The ATLAM module in this embodiment has three processing lines, labeled b1, b2 and b3 respectively. Among them, b1 is the shallow adaptive line for icing feature extraction; b2 is the medium adaptive line for icing feature extraction; and b3 is the deep adaptive line for icing feature extraction.

[0113] like Figure 2 As shown, the initial feature map F9 of the multi-scale icing is used as the input feature map of ATLAM. For ease of explanation, F9 is denoted as Q0 here.

[0114] In the shallow adaptive icing feature extraction route b1, Q0 is convolved using a dilated convolution with an inflation rate of 5 and a size of 3×3 to obtain the adaptive icing location attention map Q1. Then, Q1 is convolved using an adaptive variable stride convolution to obtain the adaptive icing location attention map Q2.

[0115] The principle of adaptive variable stride convolution is to obtain the initial sampling coordinates based on the size of the convolution kernel and dynamically determine the stride S of the dynamic deformation convolution, as shown in the following formula:

[0116]

[0117] Where m represents the kernel size. This indicates that the value of v is rounded down; S is the stride of the dynamically deformable convolution during the convolution process; [z] odd denoted by z, which is the closest odd number to z, c represents the number of channels in the output feature map, and β and θ are both hyperparameters.

[0118] In practical applications, μ can be set to 2, for example.

[0119] and It is the initial sampled coordinate data calculated based on the m-value. First, according to and A numerically generated tensor is used as the sampling coordinates for the standard convolution kernel; subsequently, the tensor is flattened into a one-dimensional vector, and then... The numerical values ​​are defined progressively, and all tensors are sequentially converted into one-dimensional vectors. The sampling coordinates of the irregular convolution kernel are determined sequentially, and finally the complete sampling coordinates are output.

[0120] In the adaptive icing feature extraction of line b1, Q0 is convolved with a dilated convolution of size 3×3 and a dilation rate of 7 to obtain the adaptive icing location attention map Q3. Then, Q3 is convolved with an adaptive variable stride convolution to obtain the fourth adaptive icing location attention map Q4.

[0121] In the deep adaptive icing feature extraction of line b3, Q0 is convolved using a dilated convolution with an inflation rate of 9 and a size of 3×3 to obtain the adaptive icing location attention map Q5. Then, Q5 is convolved using an adaptive variable stride convolution to obtain the adaptive icing location attention map Q6.

[0122] Merge Q2, Q4, and Q6 along the channel dimension (i.e.) Figure 4 The concat operation in the model is used, and the number of channels is adjusted using a regular convolution of size 1×1 to obtain the adaptive attention map Q7 for the icing location. Then, Q7 is subjected to lateral average pooling (i.e., ...). Figure 4 X_Avg Pool operation in the middle) and vertical average pooling (i.e. Figure 4 The Y_Avg Pool operation in the code pools Q7 to obtain the vertical and horizontal outputs, which are then normalized using Batch Normalization (BN) to obtain the adaptive icing position attention maps Q8 and Q9, respectively. Next, the vertical and horizontal outputs are fused along the channel dimension to obtain the adaptive icing position attention map Q10. Finally, Q10 is element-wise multiplied with Q0 (i.e., ...). Figure 4 (Multiplication operation in the middle) to obtain the adaptive line icing position attention map Q11.

[0123] In another embodiment, constructing the icing location determination model includes:

[0124] S20: Construct at least three prediction heads and a filtering module connected to the at least three prediction heads to form the icing location determination model based on the at least three prediction heads and the filtering module;

[0125] Specifically, at least three prediction heads are used to obtain second multi-scale icing depth feature maps with different scales output by the icing depth feature extraction model at different stages, and to perform regression convolution and classification convolution operations on the obtained second multi-scale icing depth feature maps to obtain sub-predicted icing location feature maps. The filtering module is used to calculate and filter the location prediction information of the icing area in the transmission line diagram based on all the sub-predicted icing location feature maps.

[0126] Furthermore, the icing location determination model is invoked to perform prediction processing on multiple second-scale icing depth feature maps, including:

[0127] S401: Using each of the prediction heads, predict the area where the transmission line is located by using the obtained second multi-scale icing depth feature map, and calculate the confidence level of the presence of icing targets in each prediction area, and associate the confidence level with the corresponding prediction area;

[0128] S402: The filtering module merges the sub-predicted icing location feature maps marked with the predicted region and confidence level output by each prediction head, and uses the non-maximum suppression algorithm and the preset cross-union ratio threshold to calculate and filter the predicted region and confidence level, and determines the target predicted region with the highest confidence level and the degree of mutual overlap is lower than the cross-union ratio threshold.

[0129] S403: The filtering module marks the target prediction region on the merged feature map.

[0130] For example, continue to combine Figure 2 As shown, the structure of the ice location determination model in this embodiment is as follows: Figure 2 As shown, there are three prediction heads (the actual number can be more or less, depending on the specific configuration). When applying the icing location determination model, regression convolution and classification convolution operations are performed on feature map F17 using the prediction heads to obtain the icing location feature map F24. The regression operation is used to predict the geometric parameters of the bounding box covering the transmission line area on F24, namely the center point coordinates (x1, y1), width w1, and height h1; the classification operation calculates the confidence (probability value) of the presence of an icing target within each prediction box and associates this confidence value with the corresponding prediction box.

[0131] The same regression and classification convolution operations are performed on feature map F20 using the prediction head to generate the icing location feature F25. The regression operation outputs the center coordinates (x2, y2), width w2, and height h2 of the predicted bounding box on F25; the classification operation similarly calculates the confidence score of the presence of icing within the corresponding bounding box and associates it with the predicted bounding box.

[0132] The prediction head performs regression and classification convolution operations on feature map F23 to output prediction head ice location feature map F26. The regression operation determines the geometric parameters (x3, y3, w3, h3) of the prediction box on F26; the classification operation calculates the confidence of ice presence within the box and associates it with the prediction box.

[0133] Finally, all predicted bounding boxes on the three feature maps F24, F25, and F26 are merged and filtered using the Non-Maximum Suppression (NMS) algorithm. NMS selects the optimal set of predicted bounding boxes with the highest confidence and a degree of overlap below the threshold, based on the confidence level of the predicted bounding boxes and a set Intersection over Union (IoU) threshold. This optimal set is then marked on the merged feature map, forming the predicted icing location feature map F27. The final selected predicted bounding box on F27 is defined by its center coordinates (x4, y4), width w4, and height h4. Its associated confidence value represents the probability of icing at that location. When this confidence level exceeds a preset threshold, it can be determined that icing exists at the corresponding transmission line location.

[0134] Specifically, the regressive convolution described above uses multi-level convolutional kernels to encode features, transforming the geometric representation of the icing target on the transmission line into a numerical output. This module employs channel dimension parameter recombination technology to generate a four-dimensional vector containing the target center point (normalized coordinates based on grid cells), width, and height (scale coefficients relative to the original image size). These values ​​are constrained to the (0,1) interval using the Sigmoid function, constructing a differentiable mapping relationship with the image space coordinate system, providing a geometric transformation basis for the subsequent decoder.

[0135] The Intersection over Union (IoU) ratio is the area of ​​the intersection between the predicted region and the actual region divided by the area of ​​the union.

[0136] The classification convolution establishes class-sensitive feature responses in the channel dimension through spatial correlation analysis of feature maps. This component utilizes a 1×1 convolution kernel to achieve dimensionality reduction of high-dimensional features and combines it with a sigmoid function to output multi-class independent probabilities, rather than the mutually exclusive classification mode of traditional softmax. For the unique texture features of icing on transmission lines, this structure strengthens the weights of key channels through an attention mechanism, effectively improving the ability to filter false positives.

[0137] As the final decision layer in the target detection process, the prediction head synchronously parses the spatial semantic information of the feature map through a parallel dual-branch architecture (regression / classification). Its output tensor interleaves and encodes multiple candidate box parameters for each grid cell in the channel dimension, including: 5-dimensional geometric parameters (x, y, w, h, confidence) and an N-dimensional class probability vector. This multi-scale prediction mechanism achieves dense coverage prediction of icy targets of different sizes through hierarchical feature fusion.

[0138] As discussed above, this embodiment employs a bounding box prediction method for prediction. Specifically, each feature map grid cell is associated with a preset number of anchor frame templates, and the regression branch predicts the coordinate correction of each anchor frame: the center offset Δx / Δy is constrained by a sigmoid function to achieve precise positioning within the grid, and the width and height scaling factors are transformed through an exponential function to maintain the numerical stability of the size prediction. This mechanism, by decoupling coordinate prediction from prior anchor frame knowledge, can significantly improve the positioning accuracy of small-sized insulating clothing targets.

[0139] The predicted confidence score reflects the probability of a valid target existing within the current anchor frame (Objectness Score). After element-wise multiplication with the classification probability matrix, the final detection confidence score is generated. This design effectively alleviates the problem of misidentification between power transmission line icing and the background in power transmission line scenarios by decoupling the target existence determination from the fine-grained classification task.

[0140] In this embodiment, the preset thresholds typically refer to the classification confidence threshold and the target intersection-over-union (IoU) threshold. Specifically, the classification confidence threshold (e.g., 0.25 or 0.5) is used to filter low-confidence prediction boxes: a prediction box is considered valid only if the prediction score for the icing category at a certain location (e.g., the probability value corresponding to the "icing exists" category) is higher than this threshold; while the IoU threshold (e.g., 0.45) is used in the non-maximum suppression (NMS) stage, where only the prediction box with higher confidence is retained when the IoU between two prediction boxes exceeds this threshold.

[0141] The Non-Maximum Suppression (NMS) algorithm is a post-processing technique for object detection, primarily used to eliminate redundant predicted boxes that overlap significantly around the same target. Its core idea is as follows: for all predicted boxes of the same category, they are sorted from highest to lowest confidence. The box with the highest confidence is used as the benchmark, and its Intersection over Union (IoU, measuring the degree of overlap between two boxes) is calculated. If the IoU of a box with the benchmark exceeds a preset threshold, they are considered to be detecting the same target, thus suppressing redundant boxes with lower confidence. This process iterates round by round, sequentially selecting the box with the highest confidence among the remaining boxes and suppressing its overlapping boxes, until all boxes have been processed. In other words, NMS effectively solves the problem of detection models repeatedly predicting the same target, ensuring concise and accurate output results.

[0142] In practical applications, the models described above in this embodiment can be, but are not limited to, YOLO models, such as the YOLOv8 model. To better illustrate the solution in this embodiment, the following description is provided in conjunction with specific examples:

[0143] All suspected icing images of transmission lines in the original transmission line icing dataset were adjusted to a uniform size of H×W×C, where H is the height of the feature map, W is the width of the feature map, and C is the number of channels of the feature map, with H being 512, W being 512, and C being 3.

[0144] The suspected icing image F0 of the transmission line is used as the input to the icing detection model and is then input into the multi-scale icing initial feature extraction module. The size of F0 is 512×512×3. First, the Conv_BN_ReLU module is applied to process F0 to obtain the first multi-scale icing initial feature map F1, with a size of 512×512×64. Then, the Conv_BN_ReLU module is applied to F1 for convolution, batch normalization, and activation function operations to obtain the second multi-scale icing initial feature map F2, with a size of 128×128×128. Next, the first MLIFE module is applied to process F2 to obtain the third multi-scale icing initial feature map F3, with a size of 128×128×256. Next, the Conv_BN_ReLU module is applied to process F4 to obtain the fourth multi-scale icing initial feature map F4, with a size of 64×64×512. F4 is then input into the second MLIFE module for further processing to obtain the fifth multi-scale icing initial feature map. The initial feature map of icing is F5, with a size of 64×64×512. Then, the Conv_BN_ReLU module is applied to F5 for convolution, batch normalization, and activation function operations to obtain the sixth multi-scale icing initial feature map F6, with a size of 32×32×512. F6 is then input into the third MLIFE module for processing to obtain the seventh multi-scale icing initial feature map F7, with a size of 32×32×512. Finally, the Conv_BN_ReLU module is applied to F7 for convolution, batch normalization, and activation function operations to obtain the eighth multi-scale icing initial feature map F8, with a size of 16×16×512. F8 is then input into the fourth MLIFE module for processing to obtain the ninth multi-scale icing initial feature map F9, with a size of 16×16×512.

[0145] The initial feature map of multi-scale line icing is input into the adaptive line icing depth feature extraction module in the transmission line icing detection network. The first ATLAM module is applied to process F9 to obtain the first adaptive line icing depth feature map F10, with a size of 16×16×512. Next, SPPF is applied to F10 to obtain the second adaptive line icing depth feature map F11, with a size of 16×16×512. After upsampling, the third adaptive line icing depth feature map F12 is obtained, with a size of 32×32×512. Finally, F7 and F12 are fused along the channel dimension to obtain the fourth adaptive line icing depth feature map. F13, with a size of 32×32×1024; then, the second ATLAM module is applied to process F13 to obtain the fifth adaptive line icing depth feature map F14, with a size of 32×32×1024. F14 is then upsampled to obtain the sixth adaptive line icing depth feature map F15, with a size of 64×64×512; finally, F5 and F15 are fused along the channel dimension to obtain the seventh adaptive line icing depth feature map F16, with a size of 64×64×1024.

[0146] F16 is input into the third ATLAM module to obtain the eighth adaptive line icing depth feature map F17, with a size of 64×64×512. Next, the Conv_BN_ReLU module is applied to F17 for convolution, batch normalization, and activation function operations to obtain the ninth adaptive line icing depth feature map F18, with a size of 32×32×512. F18 is then fused with F14 along the channel dimension to obtain the tenth adaptive line icing depth feature map F19, with a size of 32×32×1536. Subsequently, the fourth ATLAM module is applied to process F19 to obtain the eleventh adaptive line icing depth feature map. The ice depth feature map F20, with a size of 32×32×512, is obtained by applying the Conv_BN_ReLU module to perform convolution, batch normalization, and activation function operations on F20 to obtain the 12th adaptive line ice depth feature map F21, with a size of 16×16×512. Subsequently, F11 and F21 are fused along the channel dimension to obtain the 13th adaptive line ice depth feature map F22, with a size of 16×16×1024. Finally, the 5th ATLAM module is applied to process F22 to obtain the 14th adaptive line ice depth feature map F23, with a size of 16×16×512. First, the first prediction head performs regression convolution and classification convolution operations on F17 to obtain the first prediction head icing location feature map F24; the second prediction head performs regression convolution and classification convolution operations on F20 to obtain the second prediction head icing location feature map F25; the third prediction head performs regression convolution and classification convolution operations on F23 to obtain the third prediction head icing location feature map F26. The predictions from F24, F25, and F26 are then merged. Each prediction box in F24, F25, and F26 is then selected using the Non-Maximum Suppression (NMS) algorithm to identify the prediction boxes with the highest probability values ​​and no significant overlap. Finally, the final prediction box with the highest probability value and no significant overlap is labeled on the merged result of F24, F25, and F26 to obtain the fourth prediction head icing location feature map F27. Finally, the center point coordinates of the final prediction box in F27 are (x4, y4), the width of the prediction box is w4, and the height is h4. The probability value corresponding to the final prediction box is the probability that there is icing at that location. The preset threshold is set to 0.5. When the probability exceeds 0.5, it is determined that there is icing on the transmission line.

[0147] In one embodiment, the training data used when training the constructed models is specially prepared. The training data preparation process includes:

[0148] First, all suspected icing images of transmission lines in the original icing dataset were resized to a uniform size. Then, considering the limited size of the original dataset, data augmentation techniques, including but not limited to geometric transformation, color gamut transformation, sharpness transformation, noise injection, and local erasure, were applied to each suspected icing image to obtain multiple augmented images. These augmented images were then added to the dataset to ensure sufficient samples for training, validation, and testing. This enhanced the robustness of the transmission line icing detection network and reduced the model's sensitivity to subtle changes in the suspected icing images. Finally, the augmented transmission line icing image dataset was scientifically divided into training, validation, and test sets in a 6:2:2 ratio.

[0149] When training the model using training data, this application also designs a loss function called the RBIoU loss function, which is used to simultaneously optimize the position, size, confidence, and accuracy of class prediction of the bounding box. In this embodiment, the RBIoU loss function is used to train the transmission line icing detection model to accurately calculate the difference between the predicted box and the ground truth box:

[0150]

[0151] In the formula, B pred and B gt These represent the predicted bounding box and the ground truth bounding box, respectively. IoU represents the intersection-union ratio (IoU) between the predicted and ground truth bounding boxes. v is the correction factor for the aspect ratio arctangent difference; w G and h G These represent the width and height of the actual bounding box, respectively; w prod and h prod Let b and b represent the width and height of the prediction box, respectively; gt ρ(·) represents the center point of the predicted bounding box and the ground truth bounding box, respectively; ρ(·) represents the Euclidean distance between the two pixels.

[0152] The RBIoU loss function addresses the challenges of variable target size, complex backgrounds, and blurred edges in transmission line icing detection. Its most significant improvement lies in integrating multi-dimensional geometric constraints into the traditional IoU loss function: It enhances the spatial alignment of the bounding box through a center point Euclidean distance penalty term, resolving positioning errors caused by center offset in small targets (such as thin ice layers); it introduces an aspect ratio arctangent correction factor, utilizing angle differences to quantify the shape similarity between the predicted and ground truth bounding boxes, effectively addressing the regression requirements for targets with varying aspect ratios (such as icing on slender conductors); and it adaptively balances overlap and shape optimization through a dynamic weighting mechanism, improving the bounding box's sensitivity to blurred edges in complex backgrounds and occluded scenarios. The RBIoU loss function's design combines optimization of position, shape, and overlap, significantly improving the model's positioning accuracy in situations with diverse icing area sizes, low contrast, and occlusion.

[0153] Furthermore, the parameters of each layer are trained and updated on the transmission line icing detection network. First, all neural network parameters are initialized, and the hyperparameters related to the transmission line icing detection network are set. The hyperparameters set include, but are not limited to, the number of training epochs, batch size, optimizer selection, learning rate, weight initialization method, and Dropout ratio.

[0154] After parameter initialization, the training and validation sets are divided into batches. During training, a batch of training data is input into the transmission line icing detection network each time, and the difference between the predicted bounding box and the ground truth bounding box is calculated using the RBIoU loss function to obtain the training loss value T for that batch. After completing one round of traversal of the entire training set, the validation set is input into the model in batches, and the validation loss value T_val for each batch is calculated using the RBIoU loss function as well. The validation loss is mainly used to monitor whether the model is overfitting (e.g., T_val stops decreasing or increases), and to adjust the training strategy accordingly, such as terminating training early or dynamically adjusting the learning rate. The loss is calculated based on the RBIoU loss function throughout the entire training and validation process. The optimizer automatically updates the network parameters based on the training loss T, while the validation loss T_val guides the strategy adjustment. The training process will continue for multiple rounds of iteration until the validation loss T_val tends to stabilize or converges to a low level, at which point the model training ends.

[0155] Once the training is complete, each model can be put into use to quickly and accurately locate icy areas in images.

[0156] In summary, the advantages of the solutions proposed in the above embodiments of this application are at least as follows: by designing a multi-dimensional line icing feature extraction submodule (MLIFE), and utilizing shallow, medium, and deep three-layer dilated convolution paths (dilation rates of 5, 7, and standard convolution, respectively) combined with the negative region smooth nonlinear mapping of the HSELU activation function, the problem of neuron "death" is prevented, the spatial dependence between pixels is enhanced, and pixel-level visual information is captured, significantly enhancing the pixel-level spatial dependence of icing edges under fog, haze, and heavy snowfall. Through the multi-scale dynamic deformation convolution (dilation rate 5 / 7 / 9) and adaptive stride generation mechanism of the adaptive line icing position attention submodule (ATLAM) with dynamic coordinates and a trigonometric-logarithmic hybrid stride S, sampling coordinates matching the icing morphology are dynamically generated; wherein the variable stride convolution is based on dynamically adjusting the convolution stride, and through coupling the initial coordinate tensor with a nonlinear trigonometric-logarithmic function, it solves the problem of insufficient adaptability of traditional convolution to the curved icing structure of transmission lines, and effectively solves the feature sampling offset problem in small-scale icing and shading areas. By combining the RBIoU loss function, and through the joint constraint of the aspect ratio arctangent correction factor and the Euclidean distance-position deviation exponential penalty of the center point, the false detection rate of icing and background of conductors and towers is significantly reduced, and the accuracy of icing detection (robustness in severe weather + recall rate of small targets) and positioning stability (adaptability to frame deformation) in complex environments are comprehensively improved.

[0157] like Figure 5 As shown, another embodiment of the present invention also provides a deep learning-based transmission line icing detection device, comprising:

[0158] The first acquisition module is used to obtain a diagram of a power transmission line with suspected icing areas;

[0159] The first calling module is used to call the initial icing feature extraction model to process the transmission line diagram and obtain multiple first multi-scale initial icing feature maps. The multiple first multi-scale initial icing feature maps involve different feature scales and are all within the first scale range. The multiple first multi-scale initial icing feature maps are composed of intermediate results and the final result output by the model.

[0160] The second calling module is used to call the ice-covered deep feature extraction model to process multiple first multi-scale ice-covered initial feature maps to obtain multiple second multi-scale ice-covered deep feature maps. The multiple second multi-scale ice-covered deep feature maps involve different feature scales and are all within the second scale range. The multiple second multi-scale ice-covered deep feature maps are composed of intermediate results and the final results output by the model.

[0161] The third calling module is used to call the icing location determination model, and use multiple prediction heads in the icing location determination model to perform prediction processing on multiple second multi-scale icing depth feature maps respectively, so as to determine the location prediction information of the icing area in the transmission line map.

[0162] In one embodiment, the system further includes a first building module for constructing the initial feature extraction model for icing, the first building module being used for:

[0163] An initial model for processing two-dimensional image data is constructed. The initial model includes multiple convolutional processing modules and a multi-dimensional feature extraction module. The convolutional processing module is formed by at least one convolutional module, a batch normalization module, and an activation function module. The multi-dimensional feature extraction module includes three processing links. Each processing link includes a dilated convolution sub-module, a normalization sub-module, and an activation function sub-module. The activation function sub-module is used to introduce a smooth nonlinear mechanism in the negative range. The first processing link and the second processing link both include a dilation rate parameter, and the dilation rate parameters are different.

[0164] The initial model is trained to obtain the initial feature extraction model for icing.

[0165] In one embodiment, when processing the input multi-scale feature map based on the multi-dimensional feature extraction module, the process includes:

[0166] The input multi-scale feature map is processed through the first processing link to obtain the first multi-dimensional icing feature map;

[0167] The input multi-scale feature map is processed through the second processing link to obtain a second multi-dimensional icing feature map;

[0168] The first multi-dimensional icing feature map and the second multi-dimensional icing feature map are fused in the channel dimension to obtain the third multi-dimensional icing feature map.

[0169] The input multi-scale feature map is processed through the third processing link to obtain the fourth multi-dimensional icing feature map;

[0170] The third multidimensional icing feature map is multiplied element-wise with the multi-scale feature map to obtain the fifth multidimensional icing feature map.

[0171] The fifth multi-dimensional icing feature map is multiplied element-wise with the fourth multi-dimensional icing feature map to obtain the first multi-scale icing initial feature map for output.

[0172] In one embodiment, the convolutional processing module and the multi-dimensional feature extraction module are set at intervals. The first multi-scale icing initial feature map generated by the convolutional processing module or the multi-dimensional feature extraction module of the previous layer is input into the convolutional processing module or the multi-dimensional feature extraction module of the next layer for further processing. The first multi-scale icing initial feature map generated by the multi-dimensional feature extraction module is simultaneously sent to the deep icing feature extraction model for processing.

[0173] In one embodiment, a second building module is further included for constructing a deep ice feature extraction model, the second building module being used for:

[0174] Multiple adaptive icing location attention submodules, pooling submodules, upsampling submodules, feature fusion submodules, and convolution submodules are constructed. The adaptive icing location attention submodule is used to simultaneously process the obtained multi-scale icing feature map through at least three processing lines to perform feature extraction processing at different degrees. The multi-scale icing feature map includes a first multi-scale initial icing feature map or a second multi-scale deep icing feature map passed from the previous feature extraction unit.

[0175] Multiple sub-modules are connected sequentially in a preset order to form the ice-covered deep feature extraction model with a multi-layer structure;

[0176] The feature fusion submodule is used to fuse the second multi-scale icing depth feature map passed from the previous layer and the first multi-scale icing depth feature map generated by the initial icing feature extraction model on the channel, and to pass the generated fused multi-scale icing depth feature map to the next layer.

[0177] In one embodiment, processing the multi-scale icing feature map obtained based on the adaptive icing location attention submodule includes:

[0178] Each processing line performs a first convolution operation and a second convolution operation on the same multi-scale icing feature map to obtain a first adaptive icing position attention map for each processing line. The first convolution operation is a dilated convolution operation, and the second convolution operation is an adaptive variable stride convolution operation.

[0179] Multiple first adaptive icing location attention maps are fused based on the channel dimension, and horizontal pooling and vertical pooling operations are performed to obtain horizontal and vertical output results.

[0180] The horizontal and vertical output results are normalized respectively.

[0181] The normalized horizontal output and vertical output are fused based on the channel dimension to obtain the second adaptive icing position attention map;

[0182] The second adaptive icing location attention map is multiplied element-wise with the initially obtained multi-scale icing feature map to obtain the third adaptive icing location attention map that forms the second multi-scale icing depth feature map.

[0183] In one embodiment, a third building module is further included for constructing the icing location determination model, the third building module being used for:

[0184] Construct at least three prediction heads and a filtering module connected to the at least three prediction heads to form the icing location determination model based on the at least three prediction heads and the filtering module;

[0185] Specifically, at least three prediction heads are used to obtain second multi-scale icing depth feature maps with different scales output by the icing depth feature extraction model at different stages, and to perform regression convolution and classification convolution operations on the obtained second multi-scale icing depth feature maps to obtain sub-predicted icing location feature maps. The filtering module is used to calculate and filter the location prediction information of the icing area in the transmission line diagram based on all the sub-predicted icing location feature maps.

[0186] In one embodiment, the icing location determination model is invoked to predict multiple second multi-scale icing depth feature maps, including:

[0187] Each prediction head is used to predict the area where the transmission line is located based on the obtained second multi-scale icing depth feature map, and the confidence level of the presence of icing targets in each prediction area is calculated, and the confidence level is associated with the corresponding prediction area.

[0188] The filtering module merges the sub-predicted icing location feature maps marked with the predicted region and confidence level output by each prediction head, and uses a non-maximum suppression algorithm and a preset cross-union ratio threshold to calculate and filter the predicted region and confidence level, thereby determining the target predicted region with the highest confidence level and a degree of mutual overlap lower than the cross-union ratio threshold.

[0189] The filtering module is used to mark the target prediction region on the merged feature map.

[0190] Another embodiment of the present invention also provides an electronic device, comprising:

[0191] One or more processors;

[0192] Memory, configured to store one or more programs;

[0193] When the one or more programs are executed by the one or more processors, the one or more processors implement the deep learning-based transmission line icing detection method as described in any one of the above descriptions.

[0194] Furthermore, one embodiment of the present invention also provides a storage medium storing a computer program, which, when executed by a processor, implements the deep learning-based transmission line icing detection method described above. It should be understood that the various solutions in this embodiment have the corresponding technical effects in the above-described method embodiments, and will not be repeated here.

[0195] Furthermore, embodiments of the present invention also provide a computer program product, which is tangibly stored on a computer-readable medium and includes computer-readable instructions that, when executed, cause at least one processor to perform a deep learning-based transmission line icing detection method as described in the embodiments above.

[0196] It should be noted that the computer storage medium of the present invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, system, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access storage medium (RAM), a read-only storage medium (ROM), an erasable programmable read-only storage medium (EPROM or flash memory), an optical fiber, a portable compact disk read-only storage medium (CD-ROM), an optical storage medium, a magnetic storage medium, or any suitable combination thereof. In the present invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. In the present invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program configured for use by or in connection with an instruction execution system, system, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, antenna, optical fiber, RF, etc., or any suitable combination thereof.

[0197] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of protection of this application is limited to these examples; within the framework of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of one or more embodiments of this application as described above, which are not provided in detail for the sake of brevity.

Claims

1. A deep learning-based method for detecting icing on power transmission lines, characterized in that, include: Obtain a map of the power transmission lines with suspected icing areas; The transmission line diagram is processed by calling the initial icing feature extraction model to obtain multiple first multi-scale initial icing feature maps. The multiple first multi-scale initial icing feature maps involve different feature scales and are all within the first scale range. The multiple first multi-scale initial icing feature maps are composed of intermediate results and the final result output by the model. The ice-deep feature extraction model is called to process multiple first multi-scale ice-deep feature maps to obtain multiple second multi-scale ice-deep feature maps. The multiple second multi-scale ice-deep feature maps involve different feature scales and are all within the second scale range. The multiple second multi-scale ice-deep feature maps are composed of intermediate results and the final result output by the model. The icing location determination model is invoked, and multiple prediction heads in the icing location determination model are used to perform prediction processing on multiple second multi-scale icing depth feature maps to determine the location prediction information of the icing area in the transmission line map.

2. The deep learning-based transmission line icing detection method according to claim 1, characterized in that, Constructing the initial feature extraction model for icing includes: An initial model for processing two-dimensional image data is constructed. The initial model includes multiple convolutional processing modules and a multi-dimensional feature extraction module. The convolutional processing module is formed by at least one convolutional module, a batch normalization module, and an activation function module. The multi-dimensional feature extraction module includes three processing links. Each processing link includes a dilated convolution sub-module, a normalization sub-module, and an activation function sub-module. The activation function sub-module is used to introduce a smooth nonlinear mechanism in the negative range. The first processing link and the second processing link both include a dilation rate parameter, and the dilation rate parameters are different. The initial model is trained to obtain the initial feature extraction model for icing.

3. The deep learning-based transmission line icing detection method according to claim 2, characterized in that, When processing the input multi-scale feature map based on the multi-dimensional feature extraction module, the following are included: The input multi-scale feature map is processed through the first processing link to obtain the first multi-dimensional icing feature map; The input multi-scale feature map is processed through the second processing link to obtain a second multi-dimensional icing feature map; The first multi-dimensional icing feature map and the second multi-dimensional icing feature map are fused in the channel dimension to obtain the third multi-dimensional icing feature map. The input multi-scale feature map is processed through the third processing link to obtain the fourth multi-dimensional icing feature map; The third multidimensional icing feature map is multiplied element-wise with the multi-scale feature map to obtain the fifth multidimensional icing feature map. The fifth multi-dimensional icing feature map is multiplied element-wise with the fourth multi-dimensional icing feature map to obtain the first multi-scale icing initial feature map for output.

4. The deep learning-based transmission line icing detection method according to claim 2, characterized in that, The convolutional processing module and the multi-dimensional feature extraction module are set at intervals. The first multi-scale icing initial feature map generated by the convolutional processing module or the multi-dimensional feature extraction module of the previous layer is input into the convolutional processing module or the multi-dimensional feature extraction module of the next layer for further processing. The first multi-scale icing initial feature map generated by the multi-dimensional feature extraction module is simultaneously sent to the deep icing feature extraction model for processing.

5. The deep learning-based transmission line icing detection method according to claim 1 or 4, characterized in that, Constructing a deep feature extraction model for ice embankments, including: Multiple adaptive icing location attention submodules, pooling submodules, upsampling submodules, feature fusion submodules, and convolution submodules are constructed. The adaptive icing location attention submodule is used to simultaneously process the obtained multi-scale icing feature map through at least three processing lines to perform feature extraction processing at different degrees. The multi-scale icing feature map includes a first multi-scale initial icing feature map or a second multi-scale deep icing feature map passed from the previous feature extraction unit. Multiple sub-modules are connected sequentially in a preset order to form the ice-covered deep feature extraction model with a multi-layer structure; The feature fusion submodule is used to fuse the second multi-scale icing depth feature map passed from the previous layer and the first multi-scale icing depth feature map generated by the initial icing feature extraction model on the channel, and to pass the generated fused multi-scale icing depth feature map to the next layer.

6. The deep learning-based transmission line icing detection method according to claim 5, characterized in that, When processing the multi-scale icing feature map obtained based on the adaptive icing location attention submodule, the following is included: Each processing line performs a first convolution operation and a second convolution operation on the same multi-scale icing feature map to obtain a first adaptive icing position attention map for each processing line. The first convolution operation is a dilated convolution operation, and the second convolution operation is an adaptive variable stride convolution operation. Multiple first adaptive icing location attention maps are fused based on the channel dimension, and horizontal pooling and vertical pooling operations are performed to obtain horizontal and vertical output results. The horizontal and vertical output results are normalized respectively. The normalized horizontal output and vertical output are fused based on the channel dimension to obtain the second adaptive icing position attention map; The second adaptive icing location attention map is multiplied element-wise with the initially obtained multi-scale icing feature map to obtain the third adaptive icing location attention map that forms the second multi-scale icing depth feature map.

7. The deep learning-based transmission line icing detection method according to claim 1, characterized in that, Constructing the icing location determination model includes: Construct at least three prediction heads and a filtering module connected to the at least three prediction heads to form the icing location determination model based on the at least three prediction heads and the filtering module; Specifically, at least three prediction heads are used to obtain second multi-scale icing depth feature maps with different scales output by the icing depth feature extraction model at different stages, and to perform regression convolution and classification convolution operations on the obtained second multi-scale icing depth feature maps to obtain sub-predicted icing location feature maps. The filtering module is used to calculate and filter the location prediction information of the icing area in the transmission line diagram based on all the sub-predicted icing location feature maps.

8. The deep learning-based transmission line icing detection method according to claim 7, characterized in that, The icing location determination model is invoked to predict multiple second-scale icing depth feature maps, including: Each prediction head is used to predict the area where the transmission line is located based on the obtained second multi-scale icing depth feature map, and the confidence level of the presence of icing targets in each prediction area is calculated, and the confidence level is associated with the corresponding prediction area. The filtering module merges the sub-predicted icing location feature maps marked with the predicted region and confidence level output by each prediction head, and uses a non-maximum suppression algorithm and a preset cross-union ratio threshold to calculate and filter the predicted region and confidence level, thereby determining the target predicted region with the highest confidence level and a degree of mutual overlap lower than the cross-union ratio threshold. The filtering module is used to mark the target prediction region on the merged feature map.

9. A deep learning-based transmission line icing detection device, characterized in that, include: The first acquisition module is used to obtain a diagram of a power transmission line with suspected icing areas; The first calling module is used to call the initial icing feature extraction model to process the transmission line diagram and obtain multiple first multi-scale initial icing feature maps. The multiple first multi-scale initial icing feature maps involve different feature scales and are all within the first scale range. The multiple first multi-scale initial icing feature maps are composed of intermediate results and the final result output by the model. The second calling module is used to call the ice-covered deep feature extraction model to process multiple first multi-scale ice-covered initial feature maps to obtain multiple second multi-scale ice-covered deep feature maps. The multiple second multi-scale ice-covered deep feature maps involve different feature scales and are all within the second scale range. The multiple second multi-scale ice-covered deep feature maps are composed of intermediate results and the final results output by the model. The third calling module is used to call the icing location determination model, and use multiple prediction heads in the icing location determination model to perform prediction processing on multiple second multi-scale icing depth feature maps respectively, so as to determine the location prediction information of the icing area in the transmission line map.

10. An electronic device, characterized in that, include: One or more processors; Memory, configured to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the deep learning-based transmission line icing detection method as described in any one of claims 1-8.

Images