Extreme weather-oriented power transmission line insulator multi-scale icing detection method and system

Abstract

This invention discloses a multi-scale icing detection method and system for transmission line insulators under extreme weather conditions. The method includes: dividing a visible light image into pixel blocks and converting them into feature vectors with position encoding; inputting these vectors into an improved Swin Transformer integrating a Dynamic Deep Sample Generation (DDSGM) module; applying DDSGM to enhance features in a four-stage feature extraction phase; additionally fusing polarized light features in the first two stages; and outputting a multi-scale feature map. The multi-scale feature map is then input into a deformable attention unit containing DDSGM, undergoing secondary enhancement and adaptive sampling (fusing physical field information to correct for offset) to obtain enhanced features. A prediction head predicts the icing category, bounding box, and thickness; Hungarian matching is used to associate the target; and an early warning system outputs the results and a warning. This invention is the first to integrate DDSGM, optimize sampling, and combine a multi-physical quantity strategy to improve robustness to low-quality images and transparent ice layers, achieving accurate detection and early warning under extreme weather conditions.

Description

Multi-scale icing detection method and system for transmission line insulators in extreme weather conditions Technical Field

[0001] This invention belongs to the field of image processing technology, and in particular relates to a multi-scale icing detection method and system for transmission line insulators in extreme weather conditions. Background Technology

[0002] In recent years, the widespread application of drones has significantly improved the efficiency of daily inspections of power transmission lines. Currently, drones have achieved functions such as multimodal perception of thermal faults, intelligent detection of power equipment defects, and automatic identification of tower tilt, greatly enriching the means of operation and maintenance.

[0003] However, in extremely cold weather, transmission lines are highly susceptible to icing, and icing of insulators can lead to serious accidents such as flashover, tripping, or even line breakage. Therefore, icing detection has become a critical task under extreme weather, low visibility, and even drone flight restrictions, urgently requiring sensing solutions with higher robustness and scenario adaptability to ensure the safe operation of the power grid in harsh environments.

[0004] Currently, icing detection methods based on machine vision and deep learning primarily involve acquiring images and inputting them into a deep neural network for state determination. This method is intuitive, requires no modification to the insulator itself, and can acquire on-site information in real time, thus gaining widespread application. However, this method still faces several challenges:

[0005] Viewpoint shift: Due to wind or icing, the shooting angle is prone to deviating from the preset viewpoint, resulting in image deviation;

[0006] Low image quality: Outdoor images have limited resolution and clarity, and are prone to blurring, obstruction, or contamination in extreme weather conditions, which greatly increases the difficulty of detection;

[0007] Insufficient model adaptability: Existing visual models have poor robustness and generalization ability under complex field conditions, and are difficult to cope with varied icing morphologies (such as crystal textures and transparent ice layers) and background interference.

[0008] While existing technologies based on general object detection frameworks such as YOLO and DETR can achieve a certain degree of icing recognition, they do not fully consider the physical characteristics of insulator icing (such as ice crystal transparency and texture layering features), and their ability to enhance low-quality images is limited, leading to increased false detection rates in extreme weather conditions. Furthermore, traditional algorithms have high computational complexity, making it difficult to meet the real-time requirements of UAV edge devices. Therefore, there is an urgent need for a novel detection method that integrates prior physical knowledge with deep visual features to improve adaptability, robustness, and lightweight deployment capabilities in complex scenarios. Summary of the Invention

[0009] To address the above technical problems, this invention provides a multi-scale icing detection method and system for transmission line insulators in extreme weather conditions.

[0010] The technical solution adopted by this invention to solve its technical problem is:

[0011] A multi-scale icing detection method for transmission line insulators in response to extreme weather conditions, the method comprising the following steps:

[0012] S100: Acquire visible light images under extreme weather conditions, divide the visible light images into non-overlapping pixel blocks, map them into feature vectors through linear transformation and add position encoding;

[0013] S200: The preprocessed image is input into the improved Swin Transformer backbone network. Within the four feature extraction stages of this network, the Dynamic Deep Sample Generation Module (DDSGM) is integrated sequentially. By applying the transparency guidance, texture enhancement, and multi-scale fusion techniques of DDSGM in each stage, the features of the current level are initially enhanced. In the first two stages, polarized light image features are additionally fused to output a set of pre-enhanced multi-scale feature maps.

[0014] S300: Input the multi-scale feature map into the deformable attention unit that integrates the DDSGM module. In each unit, the input feature map is first enhanced by DDSGM in a secondary enhancement process oriented towards attention sampling. Then, the enhanced features are input into the deformable attention mechanism for adaptive sampling. At the same time, physical field perception information is fused to correct the sampling offset and obtain the enhanced features.

[0015] S400: It uses the prediction head structure to predict the icing category, regress the bounding box, and estimate the ice thickness of the enhanced features; it uses the Hungarian matching algorithm to associate the prediction with the real target, and outputs the icing detection results and early warning information based on the dynamic risk index and adaptive threshold through the early warning decision system.

[0016] Preferably, the transparency guidance in S200 targets transparent areas in the image, guiding the deep learning network to enhance the details of the transparent areas by strengthening the transparency contrast of the image, specifically:

[0017] ;

[0018] in, It is the enhanced feature map; It is the feature map of the original input; It is a transparency-guided feature map at the extraction point; These are learnable weight coefficients that are dynamically adjusted based on the proportion of transparent regions of the features.

[0019] In S200, texture enhancement is achieved by processing input features using a deep learning model to strengthen texture information. Specifically:

[0020] ;

[0021] in, It is a feature map after texture enhancement; This indicates that the input image is processed through a convolutional neural network. Process the data and extract texture features;

[0022] In S200, multi-scale fusion refers to fusing feature maps from different scales through weighted summation, specifically:

[0023] ;

[0024] in, It is a feature map after multi-scale fusion; It is the input feature map at the i-th scale; These are the weighting coefficients of the corresponding feature maps, obtained through training. The total number of scales participating in the integration.

[0025] Preferably, the four-level feature extraction process of S200 is as follows:

[0026] In the first stage, Swing Block and Patch Merging are used, and the texture enhancement function of DDSGM is applied to output a feature map with a size of H / 4×W / 4×96.

[0027] In the second stage, the Swing Block and Patch Merging are used, and the multi-scale fusion function of DDSGM is applied to output a feature map with a size of H / 8×W / 8×192.

[0028] In the third stage, the Swing Block and Patch Merging are used, and the transparency guidance function of DDSGM is applied to output a feature map with a size of H / 16×W / 16×384.

[0029] In the fourth stage, the Swing Block is used, and the texture enhancement and multi-scale fusion functions of DDSGM are applied to output a feature map with a size of H / 32×W / 32×768.

[0030] Preferably, in S200, polarized light image features are additionally fused in the first two stages. Specifically, a polarized light channel is introduced in the first and second stages, the polarized light image is aligned with the visible light image, and features are mixed using 1×1 convolution.

[0031] Fusion feature = weight × visible light feature + (1 - weight) × polarized light feature.

[0032] Preferably, in S300, the input feature map is first subjected to a secondary enhancement process oriented towards attention sampling using DDSGM, and its output feature map is... It is obtained by weighted fusion of texture, transparency, and multi-scale features, specifically:

[0033] ;

[0034] ;

[0035] ;

[0036] in, This is the texture map after convolution enhancement. Enhancement map for transparent areas, This is a fusion image of images at different scales. , , These are trainable weights.

[0037] Preferably, in S300, the enhanced features are input into a deformable attention mechanism for adaptive sampling, while physical field perception information is fused to correct the sampling offset, resulting in enhanced features, including:

[0038] Feature map after secondary enhancement of DDSGM Each pixel on The deformable attention mechanism generates K sampling offsets for it. and the corresponding attention weights The output features are calculated using the following aggregation formula. :

[0039] ;

[0040] Using the physical field sensing information composed of polarization map T, gradient map G, and entropy map H, the sampling offset is... Perform spatial adaptive correction:

[0041] When pixel polarization degree at At that time, it was determined to be a high transparency area, based on the store's transparency gradient. Enhanced correction of the offset:

[0042] ;

[0043] When pixel Information entropy at the location hour, If the information entropy threshold is used to identify a region with highly complex texture, the number of attention heads K at that point is dynamically increased to improve the sampling density.

[0044] ;

[0045] The feature map obtained after the above adaptive sampling and offset correction The features are then residually connected to the original input features to output the final enhanced features.

[0046] Preferably, the prediction head structure in S400 includes three parallel branches: a category prediction branch, a bounding box regression branch, and an ice thickness estimation branch.

[0047] The category prediction branch consists of two fully connected layers. The first fully connected layer maps the input features from 384 dimensions to 256 dimensions and uses the ReLU activation function to introduce non-linear expressive power to enhance the model's representation learning ability. The second fully connected layer further compresses the 256-dimensional features to N_class dimensions, which matches the number of categories. Finally, the output is transformed into a probability distribution through the Softmax activation function, as follows:

[0048] ;

[0049] in, For the predicted class probability distribution, The feature vector is input to the prediction head. and The weights and biases of the first fully connected layer. and The weights and biases for the second fully connected layer;

[0050] The bounding box regression branch maps 384-dimensional features to 256-dimensional and 4-dimensional features through two fully connected layers, uses the Sigmoid function to output normalized bounding box parameters [x, y, w, h], and has a dynamic adjustment mechanism: when the width w > 0.3 or the height h > 0.4 is detected, the width and height are automatically limited to twice the median width and height of the predicted boxes in the current batch.

[0051] The ice thickness estimation branch maps the 384-dimensional features to 128-dimensional and 1-dimensional features through two fully connected layers. It uses the ReLU activation function and combines this with a physical correction formula to calculate the thickness value. The physical correction formula is as follows:

[0052] ;

[0053] in, To estimate the ice thickness, The maximum response value in the feature map reflects the intensity of the optical characteristics of the ice layer. This is the freezing point of water. For real-time ambient temperature, and This is the physical correction factor.

[0054] Preferably, in S400, the Hungarian matching algorithm is used to associate the predicted target with the real target, specifically as follows:

[0055] Let the set of detection boxes be The set of real bounding boxes is By calculating the matching cost between pairs Construct a cost matrix, where the cost function Employing IoU loss or L2 distance; finding the optimal match This makes the total cost Minimize the detection box while ensuring that each detection box and ground truth box are matched at most once; finally, output the set of matching pairs as well as the unmatched redundant detection boxes or missed ground truth targets, thus completing the efficient data association between the prediction results and the ground truth annotations.

[0056] Preferably, in S400, the early warning decision system outputs icing detection results and early warning information based on a dynamic risk index and an adaptive threshold, including:

[0057] The system receives the probability distribution of icing categories, bounding box coordinates, ice thickness estimates, and raw confidence scores of each result from the detection network. The raw confidence scores are then calibrated using a temperature compensation formula, specifically:

[0058] ;

[0059] in, Indicates the original confidence level. The calibrated confidence level;

[0060] Entering the multi-level risk assessment stage, the dynamic risk index is calculated. Specifically:

[0061] ;

[0062] in, This represents the probability of severe icing in the probability distribution of icing categories. The icing growth rate is obtained by matching historical databases;

[0063] when Exceeding the adaptive threshold When a warning event is triggered, among which This is the historical average risk value for the current month. The standard deviation is used; a three-level response plan is initiated based on the risk index range: when When ∈[0,1.50), a regular inspection is triggered. When the range is ∈ [1.50, 2.50), schedule the UAV for re-inspection and prepare de-icing equipment. The circuit breaker remote tripping protocol is activated when the temperature reaches ≥2.50 or ice flashover characteristics are detected.

[0064] All decision-making processes are written to the blockchain log in real time, and the power grid stability changes under different disposal schemes are simulated through a digital twin system. The final output early warning information contains a standardized structure of six elements: <tower location code, icing level, risk index, confidence level, recommended measures, and time window>.

[0065] A multi-scale icing detection system for transmission line insulators designed for extreme weather conditions includes:

[0066] The image processing module is used to divide the input visible light image into non-overlapping pixel blocks, map them into feature vectors through linear transformation, and add position encoding;

[0067] The preliminary enhancement module is used to input the preprocessed image into the improved Swing Transformer backbone network. Within the four feature extraction stages of this network, the Dynamic Deep Sample Generation (DDSGM) module is integrated sequentially. By applying the transparency guidance, texture enhancement, and multi-scale fusion techniques of DDSGM in each stage, the features of the current level are initially enhanced. In the first two stages, polarized light image features are additionally fused to output a set of pre-enhanced multi-scale feature maps.

[0068] The secondary enhancement module is used to input multi-scale feature maps into the deformable attention unit that integrates the DDSGM module. In each unit, the input feature map is first enhanced by DDSGM in a secondary enhancement process oriented towards attention sampling. Then, the enhanced features are input into the deformable attention mechanism for adaptive sampling. At the same time, physical field perception information is fused to correct the sampling offset, and the enhanced features are obtained.

[0069] The icing detection module is used to predict icing category, regress bounding boxes, and estimate ice thickness based on enhanced features using a prediction head structure. It employs the Hungarian matching algorithm to associate the predictions with the real targets and outputs icing detection results and early warning information through an early warning decision system based on a dynamic risk index and adaptive threshold.

[0070] The aforementioned multi-scale icing detection method and system for transmission line insulators in extreme weather conditions, through the combination of a dynamic deep sample generation module (DDSGM) and a deformable attention mechanism, improves the accuracy of feature extraction of ice crystal areas through transparency guidance, texture enhancement and multi-scale fusion technology, and achieves efficient identification of transparent ice layers and low-quality images in complex environments. Ultimately, it realizes accurate perception and real-time early warning of icing under extreme weather conditions. Attached Figure Description

[0071] Figure 1 is a flowchart of a multi-scale icing detection method for transmission line insulators in extreme weather conditions according to an embodiment of the present invention.

[0072] Figure 2 is a flowchart of a multi-scale icing detection method for transmission line insulators in another embodiment of the present invention for extreme weather conditions. Detailed Implementation

[0073] To enable those skilled in the art to better understand the technical solution of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings.

[0074] In one embodiment, as shown in Figures 1 and 2, a multi-scale icing detection method for transmission line insulators in extreme weather conditions includes the following steps:

[0075] S100: Acquire visible light images under extreme weather conditions, divide the visible light images into non-overlapping pixel blocks, map them into feature vectors through linear transformation, and add position encoding.

[0076] Specifically, the input visible light image (e.g., 1920×1080 resolution) is first divided into non-overlapping 4×4 pixel patches. Each patch is mapped to a 96-dimensional feature vector through a linear transformation (similar to using 96 numbers to describe the initial features of an ice crystal). Then, a positional encoding based on relative coordinates is added, that is, each patch is located to ensure that the computer can remember its original position.

[0077] S200: The preprocessed image is input into the improved Swing Transformer backbone network. Within the four feature extraction stages of this network, the Dynamic Deep Sampling Generation Module (DDSGM) is integrated sequentially. By applying the transparency guidance, texture enhancement and multi-scale fusion techniques of DDSGM in each stage, the features of the current level are initially enhanced. In the first two stages, polarized light image features are additionally fused to output a set of pre-enhanced multi-scale feature maps.

[0078] Specifically, to optimize image perception capabilities in complex and low-quality visual environments, innovative improvements are made to the original SwinTransformer structure by introducing the Dynamic Deep Sample Generation (DDSGM) module. This module is an innovative approach to image processing and feature extraction, primarily used to enhance low-contrast regions in images, improve the detection accuracy of texture details, and effectively identify targets in complex scenes. By introducing three techniques—transparency guidance, texture enhancement, and multi-scale fusion—this module not only improves the visual quality of images but also enhances the perception capabilities of the deep learning model.

[0079] S300: Input the multi-scale feature map into the deformable attention unit that integrates the DDSGM module. In each unit, the input feature map is first enhanced by DDSGM in a secondary enhancement process oriented towards attention sampling. Then, the enhanced features are input into the deformable attention mechanism for adaptive sampling. At the same time, physical field perception information is fused to correct the sampling offset and obtain the enhanced features.

[0080] S400: It uses the prediction head structure to predict the icing category, regress the bounding box, and estimate the ice thickness of the enhanced features; it uses the Hungarian matching algorithm to associate the prediction with the real target, and outputs the icing detection results and early warning information based on the dynamic risk index and adaptive threshold through the early warning decision system.

[0081] Specifically, the aforementioned multi-scale icing detection method for transmission line insulators in extreme weather conditions addresses the shortcomings of existing visual detection methods in adapting to complex scenes and feature extraction from low-quality images by integrating a Dynamic Deep Sample Generation Module (DDSGM) and a deformable attention mechanism. Specifically, the method introduces the DDSGM module into the four-level feature extraction process of the Swin Transformer backbone network, enhancing image transparency guidance, texture details, and multi-scale feature fusion, thereby improving the accuracy of detecting ice crystals, ice layers, and transparent ice layers. Adaptive feature sampling through the deformable attention mechanism enables the model to achieve efficient target recognition in low-contrast, occluded, and complex backgrounds. Furthermore, this invention further optimizes the feature extraction and ice crystal detection processes by incorporating physical priors (such as polarization maps and temperature differences), ultimately achieving accurate perception and real-time early warning of icing under extreme weather conditions.

[0082] In one embodiment, the transparency guidance in S200 uses a guidance mechanism based on local image features to distinguish the features of the transparent parts of the image from those of the background, making the texture and structure of the transparent areas more prominent. The core idea is to guide the deep learning network to enhance the details of the transparent areas by strengthening the image's transparency contrast, specifically:

[0083] ;

[0084] in, It is the enhanced feature map; It is the feature map of the original input; It is a transparency-guided feature map at the extraction point; These are learnable weight coefficients that are dynamically adjusted based on the proportion of transparent regions of the features.

[0085] The purpose of texture enhancement in S200 is to strengthen the texture information in an image, especially to sharpen low-contrast, blurred, or noisy areas, ensuring accurate extraction of detail. Deep learning methods, especially convolutional neural networks (CNNs), can perform multi-level texture enhancement on images. Its core is to process input features through a deep learning model to enhance texture information, specifically:

[0086] ;

[0087] in, It is a feature map after texture enhancement; This indicates that the input image is processed through a convolutional neural network. Processing is performed to extract texture features, thereby improving the recognizability of images against complex backgrounds;

[0088] Multi-scale fusion in S200 extracts information from feature maps at different scales and merges them using weighted averaging or concatenation, helping the network to understand image content more comprehensively, especially effective when processing high-resolution and low-resolution images. It not only improves the representation of details in images but also enhances the model's ability to detect targets at different levels and sizes. In this invention, feature maps from different scales are fused through weighted summation, specifically:

[0089] ;

[0090] in, It is a feature map after multi-scale fusion; It is the input feature map at the i-th scale; These are the weighting coefficients of the corresponding feature maps, obtained through training. The total number of scales participating in the integration.

[0091] In one embodiment, the four-level feature extraction process of S200 is as follows:

[0092] In the first stage, the Swing Block (Swin module) and Patch Merging (image patch merging) are used, and the texture enhancement function of DDSGM is applied to output a feature map with a size of H / 4×W / 4×96.

[0093] In the second stage, the Swing Block and Patch Merging are used, and the multi-scale fusion function of DDSGM is applied to output a feature map with a size of H / 8×W / 8×192.

[0094] In the third stage, the Swing Block and Patch Merging are used, and the transparency guidance function of DDSGM is applied to output a feature map with a size of H / 16×W / 16×384.

[0095] In the fourth stage, the Swing Block is used, and the texture enhancement and multi-scale fusion functions of DDSGM are applied to output a feature map with a size of H / 32×W / 32×768.

[0096] Specifically, the four-level feature extraction process is shown in Table 1.

[0097] Table 1

[0098]

[0099] As shown in Table 1 above, the first level observes a small 4×4 pixel window and outputs a feature map of 1 / 4 of the original size (e.g., 480×270 pixels), which can reveal the tiny reflective points of ice crystals and the texture of the insulator surface; the second level merges four adjacent small squares into one new square and outputs a feature map of 1 / 8 of the original size (e.g., 240×135 pixels), which can reveal the formation of a network structure in the ice layer and the boundary between transparent and opaque ice; the third level expands the observation window to 8×8 pixels and outputs a feature map of 1 / 16 of the original size (e.g., 120×68 pixels), which can reveal the ice distribution of the entire insulator string and the overall trend of icicle drooping; and the fourth level integrates all local information to determine which areas are most severely iced and whether there are areas at risk of breakage, and outputs a feature map of 1 / 32 of the original size (e.g., 60×34 pixels).

[0100] In one embodiment, in S200, polarized light image features are additionally fused in the first two stages. Specifically, a polarized light channel is introduced in the first and second stages, the polarized light image is aligned with the visible light image, and features are mixed using 1×1 convolution. This enables more accurate identification of transparent ice.

[0101] Fusion feature = weight × visible light feature + (1 - weight) × polarized light feature.

[0102] Furthermore, after completing the multi-stage feature extraction of S200, S300 introduces a deformable attention mechanism based on DDSGM module fusion, which aims to further enhance the discriminative features of ice crystal regions, especially in images with challenges such as transparent ice layers, complex background textures and strong light reflection, to ensure that the target region can be accurately identified and located.

[0103] This module first receives multi-scale feature maps from four layers of the Swin Transformer backbone network (including a high-resolution 480×270×96 feature map for capturing reflections from fine ice crystals, a medium-resolution 240×135×192 feature map for analyzing ice texture structure, a low-resolution 120×68×384 feature map for recognizing the overall shape of ice crystals, and a global 60×34×768 feature map for integrating comprehensive information). These are then sequentially fed into the deformable attention unit integrating the DDSGM module. In each feature map, the DDSGM first performs transparency guidance, texture enhancement, and multi-scale fusion operations on the local region to improve feature recognition before handing it over to the deformable attention module for adaptive sampling.

[0104] In one embodiment, S300 first performs a secondary enhancement process on the input feature map using DDSGM with attention-oriented sampling, and its output feature map... It is obtained by weighted fusion of texture, transparency, and multi-scale features, specifically:

[0105] ;

[0106] ;

[0107] ;

[0108] in, This is the texture map after convolution enhancement. Enhancement map for transparent areas, This is a fusion image of images at different scales. , , These are trainable weights.

[0109] In one embodiment, in step S300, the enhanced features are input into a deformable attention mechanism for adaptive sampling, while physical field awareness information is fused to correct the sampling offset, resulting in enhanced features, including:

[0110] Feature map after secondary enhancement of DDSGM Each pixel on The deformable attention mechanism generates K sampling offsets for it. and the corresponding attention weights The output features are calculated using the following aggregation formula. :

[0111] ;

[0112] Using the physical field sensing information composed of polarization map T, gradient map G, and entropy map H, the sampling offset is... Perform spatial adaptive correction:

[0113] When pixel polarization degree at At that time, it was determined to be a high transparency area, based on the store's transparency gradient. Enhanced correction of the offset:

[0114] ;

[0115] When pixel Information entropy at the location hour, If the information entropy threshold is used to identify a region with highly complex texture, the number of attention heads K at that point is dynamically increased to improve the sampling density.

[0116] ;

[0117] The feature map obtained after the above adaptive sampling and offset correction The features are then residually connected to the original input features to output the final enhanced features.

[0118] Specifically, this mechanism ensures that the density and location distribution of sampling points more closely match the actual ice crystal distribution area, enabling accurate characterization of irregular, transparent, multi-layered ice structure.

[0119] In one embodiment, the prediction head structure in S400 includes three parallel branches: a category prediction branch, a bounding box regression branch, and an ice thickness estimation branch.

[0120] The prediction head structure is shown in Table 2 below:

[0121] Table 2

[0122]

[0123] As can be seen from Table 2 above, this structure mainly predicts three core pieces of information: the first part is the hazard level judgment, which takes the ice layer features extracted in the previous steps as input and judges whether it is safe or whether the icing is severe through a neural network; the second part is the ice floe location definition, which mainly outputs four key numbers [center X coordinate, center Y coordinate, frame width, frame height] and performs automatic calibration; the third part is the estimation of ice layer thickness based on the reflection intensity calculation.

[0124] The category prediction branch consists of two fully connected layers. The first fully connected layer maps the input features from 384 dimensions to 256 dimensions and uses the ReLU activation function to introduce non-linear expressive power to enhance the model's representation learning ability. The second fully connected layer further compresses the 256-dimensional features to N_class dimensions, which matches the number of classes. Finally, the output is transformed into a probability distribution through the Softmax activation function, thus achieving the multi-class classification task. Specifically:

[0125] ;

[0126] in, For the predicted class probability distribution, The feature vector is input to the prediction head. and The weights and biases of the first fully connected layer. and The weights and biases for the second fully connected layer;

[0127] The bounding box regression branch maps 384-dimensional features to 256-dimensional and 4-dimensional features through two fully connected layers. It uses the Sigmoid function to output normalized bounding box parameters [center x, center y, width w, height h]. The center point coordinates and width and height are compressed to the range of 0-1 to adapt to input images of different resolutions. A dynamic adjustment mechanism is set up to correct abnormally large bounding boxes: when a width w > 0.3 or a height h > 0.4 is detected, it indicates that a false detection may have occurred. The width and height are automatically limited to twice the median width and height of the predicted bounding boxes in the current batch (implemented by Python code "if (w>0.3 or h>0.4): w = min(w, 2*median_width); h = min(h, 2*median_height)"). This design maintains the algorithm's adaptability to insulators of various sizes and effectively suppresses the problem of abnormally large bounding boxes caused by extreme weather or imaging noise, so that the positioning accuracy can still maintain a stable performance of ±3 pixels in complex environments.

[0128] The ice thickness estimation branch maps the 384-dimensional features to 128-dimensional and 1-dimensional features through two fully connected layers. It uses the ReLU activation function and combines this with a physical correction formula to calculate the thickness value. The physical correction formula is as follows:

[0129] ;

[0130] in, To estimate the ice thickness, The maximum response value in the feature map reflects the intensity of the optical characteristics of the ice layer. This is the freezing point of water. For real-time ambient temperature, and This is a physical correction factor. Temperature difference term. This quantifies the thermodynamic conditions for ice formation. The model was calibrated using over 2000 sets of field measurement data. =0.12 and The two key coefficients, 0.05, enable the system to achieve a measurement accuracy of ±1mm within the operating temperature range of -20℃ to 0℃. This design, which integrates machine vision and thermodynamic principles, effectively solves the problem of large errors in the measurement of transparent ice thickness using traditional pure vision methods.

[0131] In one embodiment, S400 employs the Hungarian matching algorithm to associate the predicted target with the real target, specifically as follows:

[0132] Let the set of detection boxes be The set of real bounding boxes is By calculating the matching cost between pairs Construct a cost matrix, where the cost function Employing IoU loss or L2 distance; finding the optimal match This makes the total cost Minimize the detection box while ensuring that each detection box and ground truth box are matched at most once; finally, output the set of matching pairs as well as the unmatched redundant detection boxes or missed ground truth targets, thus completing the efficient data association between the prediction results and the ground truth annotations.

[0133] Specifically, Hungarian matching optimization is an allocation algorithm based on maximum weight matching in a bipartite graph. Its core idea is to construct a cost matrix and find the optimal matching to minimize the total matching cost. In practice, it can be optimized through augmented path search or the dual problem of linear programming.

[0134] Finally, the early warning decision system, as the final decision-making link in the entire insulator icing detection process, transforms the icing feature data obtained in the preceding steps into executable early warning instructions through a multi-dimensional risk quantification model and an adaptive threshold mechanism. Its core is a four-layer decision architecture that integrates machine vision confidence, icing physical parameters, environmental condition data, and historical statistical patterns.

[0135] In one embodiment, S400 outputs icing detection results and early warning information based on a dynamic risk index and an adaptive threshold through an early warning decision system, including:

[0136] The system receives the probability distribution of icing categories (safe / light / heavy), bounding box coordinates, ice thickness estimates, and raw confidence levels of each result from the detection network. The raw confidence levels are then calibrated using a temperature compensation formula, specifically:

[0137] ;

[0138] in, Indicates the original confidence level. The calibrated confidence level;

[0139] Entering the multi-level risk assessment stage, the dynamic risk index is calculated. Specifically:

[0140] ;

[0141] in, This represents the probability of severe icing in the probability distribution of icing categories. The icing growth rate is obtained by matching historical databases;

[0142] when Exceeding the adaptive threshold When a warning event is triggered, among which This is the historical average risk value for the current month. The standard deviation is used; for triggered alarms, the system will further analyze the impact range by combining the power grid topology, use Dijkstra's algorithm to calculate the optimal path to the nearest de-icing resource point, and generate a three-level response plan that includes the timeliness of handling: when a blue alert is triggered When ∈[0,1.50), a regular inspection is triggered; when a yellow warning is triggered... When the value is in the range [1.50, 2.50), schedule the drone for re-inspection and prepare de-icing equipment. When a red alert is triggered... When the temperature reaches ≥2.50 or ice flashover characteristics are detected, the circuit breaker remote tripping protocol will be activated and the emergency command center will be activated.

[0143] All decision-making processes are written to the blockchain log in real time to ensure that the operations are tamper-proof. The digital twin system simulates the changes in power grid stability under different disposal schemes. The final output early warning information contains a standardized structure of six elements: <tower location code, icing level, risk index, confidence level, recommended measures, and time window>.

[0144] The final output report module encapsulates the detection results into immediately executable operation and maintenance instructions through structured data encapsulation. Its core is a standardized data packet containing six-dimensional information: First, it integrates the insulator location information output by the detection network (including tower number, phase and three-dimensional coordinates), icing level judgment results (three-level classification based on the dynamic risk index Risk), quantitative parameters (ice thickness accurate to 0.1mm, ice length and droop angle), confidence score (0-100% value after temperature-visibility multi-factor calibration), handling suggestions (specific to the de-icing method such as DC de-icing or mechanical de-icing and its priority), and time window (remaining safe time calculated according to the icing growth model). After these data are encapsulated in JSON-LD format, a human-readable natural language report is generated.

[0145] This invention is the first to integrate the Dynamic Deep Sample Generation Module (DDSGM) into the feature extraction and attention mechanism for insulator icing detection, significantly improving the system's robustness and perception capabilities against low-quality images, transparent ice layers, and complex backgrounds. Compared to the traditional Swing Transformer extraction path, DDSGM achieves triple enhancement of local texture, transparent regions, and size variations in S200, making the original feature map more fully expressive of ice crystal details. In S300, through the preprocessing of the DDSGM module and the linkage of deformable attention, spatial adaptive sampling point optimization is achieved, avoiding the omission of target information due to static sampling.

[0146] Furthermore, the multi-physical quantity-assisted migration strategy proposed in this invention (integrating the transparent map T, the information entropy map H, and polarization guidance) takes into account both geometric structure and optical features, greatly improving the model's adaptability to uncertain icing boundaries and non-uniform thickness distributions.

[0147] In one embodiment, a multi-scale icing detection system for transmission line insulators in extreme weather conditions is also provided, comprising:

[0148] The image processing module is used to divide the input visible light image into non-overlapping pixel blocks, map them into feature vectors through linear transformation, and add position encoding;

[0149] The preliminary enhancement module is used to input the preprocessed image into the improved Swing Transformer backbone network. Within the four feature extraction stages of this network, the Dynamic Deep Sample Generation (DDSGM) module is integrated sequentially. By applying the transparency guidance, texture enhancement, and multi-scale fusion techniques of DDSGM in each stage, the features of the current level are initially enhanced. In the first two stages, polarized light image features are additionally fused to output a set of pre-enhanced multi-scale feature maps.

[0150] The secondary enhancement module is used to input multi-scale feature maps into the deformable attention unit that integrates the DDSGM module. In each unit, the input feature map is first enhanced by DDSGM in a secondary enhancement process oriented towards attention sampling. Then, the enhanced features are input into the deformable attention mechanism for adaptive sampling. At the same time, physical field perception information is fused to correct the sampling offset, and the enhanced features are obtained.

[0151] The icing detection module is used to predict icing category, regress bounding boxes, and estimate ice thickness based on enhanced features using a prediction head structure. It employs the Hungarian matching algorithm to associate the predictions with the real targets and outputs icing detection results and early warning information through an early warning decision system based on a dynamic risk index and adaptive threshold.

[0152] Specific limitations regarding the multi-scale icing detection system for transmission line insulators in extreme weather conditions can be found in the limitations of the multi-scale icing detection method for transmission line insulators in extreme weather conditions described above, and will not be repeated here. Each module in the aforementioned multi-scale icing detection system for transmission line insulators in extreme weather conditions can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0153] The above provides a detailed description of the multi-scale icing detection method and system for transmission line insulators in extreme weather conditions provided by this invention. Specific examples have been used to illustrate the principles and implementation methods of this invention, and the descriptions of these embodiments are merely for the purpose of helping to understand the core ideas of this invention. It should be noted that those skilled in the art can make various improvements and modifications to this invention without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this invention.

Claims

1. A multi-scale icing detection method for transmission line insulators in extreme weather conditions, characterized in that, The method The process includes the following steps: S100: Acquire a visible light image under extreme weather conditions, divide the visible light image into non-overlapping pixel blocks, map it to a feature vector through a linear transformation, and add positional encoding; S200: Input the preprocessed image into the improved Swin... The Transformer backbone network integrates a Dynamic Deep Sample Generation (DDSGM) module sequentially within its four feature extraction stages. At each stage, DDSGM's transparency guidance, texture enhancement, and multi-scale fusion techniques are applied to initially enhance the features of the current layer. In the first two stages, polarized light image features are additionally fused, outputting a set of pre-enhanced multi-scale feature maps. In step S300, the multi-scale feature maps are input into deformable attention units integrating the DDSGM module. Within each unit, the input feature maps undergo secondary enhancement processing via DDSGM for attention-oriented sampling. The enhanced features are then input into the deformable attention mechanism for adaptive sampling, while simultaneously fusing physical field-aware information to correct sampling offsets, resulting in enhanced features. In step S300, the enhanced features are input into the deformable attention mechanism for adaptive sampling, while simultaneously fusing physical field-aware information to correct sampling offsets, resulting in enhanced features, including: the feature maps enhanced by the secondary DDSGM enhancement. Each pixel on The deformable attention mechanism generates K sampling offsets for it. and the corresponding attention weights The output features are calculated using the following aggregation formula. ： Using the physical field sensing information composed of polarization map T, gradient map G, and entropy map H, the sampling offset is... Perform spatial adaptive correction: when pixel polarization degree at When a region is identified as having high transparency, the transparency gradient at that point is used to determine its location. Enhanced correction of the offset: When pixel Information entropy at the location hour, If the information entropy threshold is used to identify a region with highly complex texture, the number of attention heads K at that point is dynamically increased to improve the sampling density. The feature map obtained after the above adaptive sampling and offset correction The features are residually connected with the original input features to output the final enhanced features; S400: the enhanced features are used to predict the icing category, regress the bounding box, and estimate the ice thickness through the prediction head structure; the Hungarian matching algorithm is used to associate the prediction with the real target, and the icing detection results and early warning information are output by the early warning decision system based on the dynamic risk index and adaptive threshold.

2. The method according to claim 1, characterized in that, In S200, transparency guidance targets transparent areas in an image, guiding the deep learning network to enhance the details of these areas by strengthening the image's transparency contrast. Specifically: ;in, It is the enhanced feature map; It is the feature map of the original input; It is a transparency-guided feature map at the extraction point; These are learnable weight coefficients that are dynamically adjusted based on the proportion of transparent regions of the features. In S200, texture enhancement is achieved by processing the input features using a deep learning model to strengthen texture information, specifically: ;in, It is a feature map after texture enhancement; This indicates that the input image is processed through a convolutional neural network. Processing is performed to extract texture features; in S200, multi-scale fusion refers to fusing feature maps from different scales through weighted summation, specifically: ;in, It is a feature map after multi-scale fusion; It is the input feature map at the i-th scale; These are the weighting coefficients of the corresponding feature maps, obtained through training. The total number of scales participating in the integration.

3. The method according to claim 2, characterized in that, The four-level feature extraction process of S200 is as follows: In the first stage, Swing Block and Patch Merging are used, and the texture enhancement function of DDSGM is applied to output a feature map with a size of H / 4×W / 4×96. In the second stage, the Swing Block and Patch Merging are used, and the multi-scale fusion function of DDSGM is applied to output a feature map with a size of H / 8×W / 8×192. In the third stage, Swing Block and Patch Merging are used, and the transparency guidance function of DDSGM is applied to output a feature map with a size of H / 16×W / 16×384. In the fourth stage, Swing Block is used, and the texture enhancement and multi-scale fusion functions of DDSGM are applied to output a feature map with a size of H / 32×W / 32×768.

4. The method according to claim 3, characterized in that, In S200, polarized light image features are additionally fused in the first two stages. Specifically, a polarized light channel is introduced in the first and second stages, the polarized light image is aligned with the visible light image and the features are fused using 1×1 convolution. Specifically, the fused feature = weight × visible light feature + (1-weight) × polarized light feature.

5. The method according to claim 4, characterized in that, In S300, the input feature map is first enhanced by DDSGM through secondary attention-oriented sampling, and its output feature map is... It is obtained by weighted fusion of the texture-enhanced feature map output by S200, the transparency-guided enhanced feature map, and the multi-scale fusion feature map.

6. The method according to claim 5, characterized in that, The prediction head structure in S400 includes three parallel branches: a class prediction branch, a bounding box regression branch, and an ice thickness estimation branch. The class prediction branch consists of two fully connected layers. The first fully connected layer maps the input features from 384 dimensions to 256 dimensions and uses the ReLU activation function to introduce non-linear expressive power to enhance the model's representation learning ability. The second fully connected layer further compresses the 256-dimensional features to N_class dimensions, which matches the number of classes. Finally, the output is transformed into a probability distribution through the Softmax activation function. ;in, For the predicted class probability distribution, The feature vector is input to the prediction head. and The weights and biases of the first fully connected layer. and The weights and biases of the second fully connected layer are defined. The bounding box regression branch maps the 384-dimensional features to 256-dimensional and 4-dimensional features through two fully connected layers, uses the Sigmoid function to output normalized bounding box parameters [x, y, w, h], and has a dynamic adjustment mechanism: when width w > 0.3 or height h > 0.4 is detected, the width and height are automatically limited to twice the median width and height of the predicted boxes in the current batch. The ice thickness estimation branch maps the 384-dimensional features to 128-dimensional and 1-dimensional features through two fully connected layers, uses the ReLU activation function, and calculates the thickness value using a physical correction formula. The physical correction formula is as follows: ;in, To estimate the ice thickness, The maximum response value in the feature map reflects the intensity of the optical characteristics of the ice layer. This is the freezing point of water. For real-time ambient temperature, and This is the physical correction factor.

7. The method according to claim 6, characterized in that, In S400, the Hungarian matching algorithm is used to associate the predicted target with the real target. Specifically, let the set of detection boxes be... The set of real bounding boxes is By calculating the matching cost between pairs Construct a cost matrix, where the cost function Employing IoU loss or L2 distance; finding the optimal match This makes the total cost Minimize the detection box while ensuring that each detection box and ground truth box are matched at most once; finally, output the set of matching pairs as well as the unmatched redundant detection boxes or missed ground truth targets, thus completing the efficient data association between the prediction results and the ground truth annotations.

8. The method according to claim 7, characterized in that, In the S400, the early warning decision system outputs icing detection results and early warning information based on a dynamic risk index and adaptive threshold. This includes receiving the icing category probability distribution, bounding box coordinates, ice thickness estimate, and the original confidence level of each result from the detection network. The original confidence level is then calibrated using a temperature compensation formula. Specifically: ;in, Indicates the original confidence level. The calibrated confidence level is then used to proceed to the multi-level risk assessment stage, where the dynamic risk index is calculated. Specifically: ;in, This represents the probability of severe icing in the probability distribution of icing categories. The icing growth rate is obtained by matching historical databases; when Exceeding the adaptive threshold When a warning event is triggered, among which This is the historical average risk value for the current month. The standard deviation is used; a three-level response plan is initiated based on the risk index range: when When ∈[0,1.50), a regular inspection is triggered. When the range is ∈ [1.50, 2.50), schedule the UAV for re-inspection and prepare de-icing equipment. The circuit breaker remote tripping protocol is activated when the temperature reaches ≥2.50 or ice flashover characteristics are detected; all decision-making processes are written to the blockchain log in real time, and the power grid stability changes under different handling schemes are simulated through a digital twin system. The final output early warning information contains a standardized structure of six elements: <tower location code, icing level, risk index, confidence level, recommended measures, and time window>.

9. A multi-scale icing detection system for transmission line insulators in extreme weather conditions, implementing the method as described in any one of claims 1-8, characterized in that, include: The image processing module is used to divide the input visible light image into non-overlapping pixel blocks, map them into feature vectors through linear transformation, and add position encoding; The initial enhancement module inputs the preprocessed image into the improved Swin Transformer backbone network. Within the four feature extraction stages of this network, the Dynamic Deep Sample Generation (DDSGM) module is sequentially integrated. By applying DDSGM's transparency guidance, texture enhancement, and multi-scale fusion techniques at each stage, the features of the current level are initially enhanced. In the first two stages, polarized light image features are additionally fused to output a set of pre-enhanced multi-scale feature maps. The secondary enhancement module inputs the multi-scale feature maps into deformable attention units integrating the DDSGM module. Within each unit, the input feature map is first enhanced by DDSGM through attention-oriented sampling, and then the enhanced features are input into the deformable attention mechanism for adaptive sampling. Simultaneously, physical field perception information is fused to correct the sampling offset, resulting in enhanced features. The icing detection module performs icing category prediction, bounding box regression, and ice thickness estimation on the enhanced features through the prediction head structure. The Hungarian matching algorithm is used to associate the prediction with the real target, and the icing detection results and warning information are output by the early warning decision system based on the dynamic risk index and adaptive threshold.

Images