Millimeter wave SAR icing thickness inversion algorithm based on deep learning

Abstract

The invention discloses a millimeter wave SAR icing thickness inversion algorithm based on deep learning, and aims to solve the problems of insufficient icing monitoring precision and poor real-time performance under complex meteorological conditions. According to the algorithm, high-quality input is provided for deep learning modeling through data preprocessing including slope distance correction, polarization denoising, back scattering characteristic extraction and data enhancement. A multi-branch deep neural network structure is adopted for feature extraction, and in combination with a Swin Transformer V2 trunk, a Gated MLP module, cross-modal attention fusion and physical consistency constraint, the adaptability of the model to millimeter wave SAR data and multi-modal input is improved. Meteorological parameters and line operation data are integrated through multi-modal data fusion, and model robustness is enhanced. Finally, an icing thickness prediction value is output through real-time reasoning, intelligent early warning is triggered in combination with confidence analysis, end-to-end real-time response is achieved, the intelligent level of safe operation of a power grid is remarkably improved, and the method has wide application value.

Description

Technical Field

[0001] The present invention relates to the technical field of radar remote sensing and power system monitoring, and in particular to a millimeter wave SAR ice thickness inversion algorithm based on deep learning. Background Art

[0002] Icing on transmission lines is a major safety hazard in power grid operations. In severe cases, it can increase conductor loads, leading to accidents such as line breakage and tower collapse, and thus affecting the stable operation of the power grid. Therefore, how to accurately and in real time monitor the ice thickness on transmission lines has become an urgent problem to be solved in the power industry. Currently, traditional ice monitoring methods mainly include ground-based weather station monitoring, manual inspections, optical and infrared remote sensing, and ultrasonic or laser detection. However, these methods all have certain limitations. For example, weather station monitoring cannot directly obtain ice thickness information on the line, manual inspections are inefficient and costly, optical and infrared remote sensing are greatly affected by weather conditions, and ultrasonic or laser detection, while highly accurate, are expensive and difficult to implement on a large scale. Therefore, finding an efficient, accurate, and applicable ice thickness monitoring method under complex meteorological conditions has become a key research direction.

[0003] Millimeter-wave synthetic aperture radar (SAR) technology, with its all-weather, all-day, and cloud-penetrating capabilities, offers a new solution for monitoring ice on transmission lines. SAR can acquire high-resolution radar imagery and, by analyzing backscatter characteristics, monitor ice on transmission lines. Currently, SAR-based ice thickness inversion methods primarily employ physical modeling, establishing microwave scattering models to analyze the propagation characteristics of electromagnetic waves in the ice layer and using empirical formulas or numerical simulations to estimate thickness. For example, researchers used Mie scattering theory to calculate the millimeter-wave backscatter coefficient under varying ice thickness conditions and fitted mathematical models to experimental data to infer ice thickness. Some studies also employ polarization information analysis, utilizing dual-polarization or fully polarized SAR data to improve the accuracy of thickness inversion.

[0004] However, physical modeling approaches have numerous limitations. Factors such as the structure of different transmission lines and the meteorological environment can affect the scattering characteristics of electromagnetic waves, resulting in poor generalization of model parameters. Physical modeling methods are computationally complex and require extensive experimental measurement data and numerical simulations, making them difficult to effectively apply in practical engineering projects. Problems such as noise and complex background interference in SAR data further reduce the accuracy of physical model inversion. To address these shortcomings, the introduction of deep learning technology has provided new insights for ice thickness inversion.

[0005] The application of deep learning in the field of SAR signal processing has made significant progress in recent years. Compared with traditional physical modeling methods, deep learning can automatically extract features in a data-driven manner, avoid the manual setting of complex physical parameters, and improve the accuracy and robustness of thickness inversion. Current ice thickness inversion methods based on deep learning mainly use models such as convolutional neural networks (CNN), recurrent neural networks (RNN), and Transformer. CNN is good at extracting spatial features from SAR images, such as backscatter intensity and texture information, while RNN and Transformer can use time series data to improve the dynamics of thickness prediction. Generative adversarial networks (GANs) are also used for data enhancement to make up for the lack of real labeled data.

[0006] In practical applications, deep learning methods first need to preprocess SAR data, including denoising, polarization information fusion, data enhancement, etc., to improve the training effect of the model. Then, deep neural networks are used to extract key features, such as backscattering features, polarization features, angular information, etc. in SAR images, and a mapping relationship between input features and ice thickness is established through training. A typical deep learning model can input SAR image data, extract features through multi-layer CNN, and finally output the predicted value of ice thickness. The Transformer model can be combined with time series SAR data to capture the dynamic changes in ice thickness and improve the timeliness and accuracy of the prediction. In order to improve the interpretability of the model, some studies have also tried to combine physical modeling with deep learning to improve the generalization ability of the model by introducing physical constraints.

[0007] Deep learning methods still have some shortcomings in SAR data analysis and ice thickness inversion. Training deep learning models relies on large amounts of annotated data, but in practice, high-quality ice thickness data is scarce and expensive to acquire. Secondly, the generalization ability of deep learning models remains limited, and accuracy may decline under varying meteorological conditions and line structures. The high computational complexity of deep learning models makes them difficult to meet the needs of real-time monitoring. Finally, because deep learning methods are primarily data-driven and lack physical interpretability, they are difficult to adopt for engineering applications in some cases. Therefore, future research directions include combining physical models with deep learning to improve model interpretability and stability; employing weakly supervised and self-supervised learning to reduce reliance on large amounts of annotated data; optimizing model structures to improve computational efficiency and enable real-time monitoring; and integrating multimodal data, such as SAR, optical remote sensing, and meteorological data, to improve the accuracy of ice thickness prediction. Summary of the Invention

[0008] To address these issues, this paper proposes a deep learning-based millimeter-wave SAR ice thickness inversion algorithm. This algorithm leverages the all-weather imaging capabilities and high-precision backscattering characteristics of millimeter-wave synthetic aperture radar (SAR), combined with a deep neural network for intelligent analysis of SAR data. This algorithm effectively overcomes the shortcomings of traditional physical modeling methods, such as unstable accuracy, high computational complexity, and poor generalization in complex environments. This method accurately inverts ice thickness on transmission lines even in adverse weather conditions such as cloud, fog, rain, and snow. Through a deep learning optimization algorithm, it improves the real-time and reliability of monitoring, providing efficient and intelligent early warning and decision-making support for the safe operation of the power grid.

[0009] In order to achieve the above object, the present invention adopts the following technical solutions:

[0010] A millimeter wave SAR ice thickness inversion algorithm based on deep learning, the millimeter wave SAR ice thickness inversion algorithm based on deep learning includes the following steps:

[0011] Step 1 Data Collection: Use the millimeter-wave SAR sensor on the drone to obtain multi-polarization VV, HH, VH, HV, and multi-time series radar echo data of the transmission line. Combined with the high-precision inertial measurement unit (IMU) and global navigation satellite system (GNSS), the drone's position, attitude, and motion information are recorded.

[0012] Step 2: Data preprocessing: The raw SAR data collected by the drone is subjected to slant range correction to eliminate geometric distortion, polarization denoising to remove coherent speckle noise, backscattering characteristics extraction to calculate polarization ratio and entropy, and data enhancement using a generative adversarial network (GAN) to ensure input data quality and stability.

[0013] Step 3: Feature extraction and modeling: Using the Swin Transformer V2 backbone structure combined with Gated MLP, cross-modal attention fusion, and a dual-path decoder, a multi-branch deep neural network is constructed to extract local and global features of millimeter-wave SAR images.

[0014] Step 4: Multimodal data fusion: Integrate millimeter-wave SAR images, meteorological data, and line operation parameters through feature-level fusion (Cross-Modal Attention) and decision-level fusion.

[0015] Step 5: Model training and optimization: L1 loss and mean square error (MSE) are used in conjunction with physical constraints to optimize model parameters, and self-supervised learning methods are introduced to improve the model's generalization ability for unlabeled data.

[0016] Step 6: Thickness inversion and real-time inference: Based on the trained deep learning model, real-time millimeter-wave SAR images and multimodal data are input to predict ice thickness on transmission lines. Confidence analysis is then used to trigger an early warning mechanism.

[0017] Step 7: System Integration and Dynamic Operation: The drone collects millimeter-wave SAR data and transmits it wirelessly to the edge computing node. GPU acceleration is used to complete deep learning inference. The results are uploaded to the power grid monitoring platform to trigger intelligent early warnings, achieving end-to-end real-time ice cover monitoring and emergency response.

[0018] Furthermore, the deep learning-based millimeter-wave SAR ice thickness inversion algorithm is based on the following system architecture: the UAV side is responsible for millimeter-wave SAR data collection and transmits the data to the ground station through wireless communication; the ground station is equipped with a high-performance computing server, which runs a deep learning model for data preprocessing, feature extraction, thickness inversion and early warning analysis; the power grid monitoring system receives the ice thickness prediction results, combines historical data for comprehensive analysis, and triggers an intelligent early warning mechanism.

[0019] Furthermore, in step 1, a UAV equipped with a millimeter-wave SAR sensor performs side-view imaging along the transmission line to obtain multi-view and multi-time series millimeter-wave SAR echo data; multi-polarization modes VV, HH, VH, and HV are used to obtain backscattered signals under different polarization states; combined with interferometric SAR or polarimetric SAR, target information is enhanced and noise interference is suppressed.

[0020] Furthermore, in step 2, slant range correction is to correct the geometric distortion of the original SAR image and convert the image from the slant range coordinate system to the ground coordinate system; polarimetric coherence filtering is used to remove coherent speckle noise and improve the stability of the target scattering characteristics; backscattering characteristic extraction is to calculate the backscattering coefficient RCS under different polarization channels and extract the scattering characteristics of the target, including polarization ratio, entropy, and uniformity.

[0021] Furthermore, in step 3, the millimeter-wave SAR ice thickness inversion network adopts a multi-branch, highly complex deep neural structure, combined with the Swin Transformer V2 backbone structure, the Gated MLP module, cross-modal attention fusion, the multi-path decoder, and the physical consistency constraint module to build a thickness inversion model that is highly adapted to the millimeter-wave SAR imaging characteristics and multi-modal input.

[0022] The input data includes three-channel millimeter-wave SAR images and synchronized meteorological parameters and transmission line operation data. The image data is encoded into low-dimensional feature blocks through the Patch Embedding module for subsequent processing by the Transformer module.

[0023] The backbone feature extraction module uses a two-stage Swin Transformer V2 encoding block to extract hierarchical image features. Each stage contains the following components: Window-based Multi-head Self-Attention to calculate attention in a local window; LayerScale module to adjust the residual path stability; Shifted Window strategy to enhance information transfer between regions; residual connection and normalization operations;

[0024] The deep reasoning module introduces the ST-Transformer structure to model cross-frame feature correlations in time series or polarization domains;

[0025] The decoder adopts a dual-path upsampling structure. The model uses two parallel decoding paths: Path 1 is a lightweight convolutional decoder with layer-by-layer upsampling and skip connections; Path 2 is a Transformer-based attention decoder. The two decoding paths are finally fused at the intermediate layer and fed into the output module.

[0026] The multi-head output structure includes: Head1, Head2, Head3;

[0027] Head1 is thickness prediction, and the output tensor size is H×W, representing the pixel-level ice thickness;

[0028] Head2 is the uncertainty estimate, output Represents the confidence interval bounds for thickness prediction, modeled using MCDropout or Bayesian estimation;

[0029] Head3 is a physical consistency decoder that feeds the thickness prediction result into the electromagnetic scattering affine layer to calculate the residual loss between the theoretical backscattering model:

[0030]

[0031] Where σ(x) is the uncertainty estimate of the model output; is the spatial dimension of the output, H and W represent the height and width of the output feature map, respectively, corresponding to the spatial resolution of the millimeter-wave SAR image; x is the input data, i.e., the millimeter-wave SAR image; is the physical consistency loss; N is the total number of samples or pixels; σ pred (h i ) is the backscattering coefficient predicted by the model; σ theory (hi ) is the theoretical backscattering coefficient; h i Ice thickness of the i-th sample; (σ pred (h i )-σ theory (h i )) 2 is the square of the physical consistency error of a single sample.

[0032] Furthermore, in step 4, the feature-level fusion process involves mapping meteorological data and operating parameters through independent MLPs and then inputting them into Cross-Modal Attention as key-value pairs to align with image features:

[0033] F fusion =Attention(Q=F img W q ,K=F env W k ,V=F env W v )

[0034] The fused output is concatenated with the image backbone features for subsequent high-level reasoning;

[0035] Where, F fusion is the fused feature representation; Attention is the attention mechanism function; Q = F img W q In this example, Q is the query matrix, F img is the image feature; W q is the query projection matrix; K = F env W k In this example, K is the bond matrix, F env is the environmental data feature, W k is the key projection matrix; V = F env In W, V is the value matrix, W v is the value projection matrix.

[0036] Furthermore, in step 4, the decision-level fusion process is to train two independent neural networks, one for SAR data and the other for multimodal data, and fuse the final prediction results by weighted averaging;

[0037] At the same time, the Gated MLP module is introduced to replace part of the convolution structure. Its basic unit is:

[0038] Output=(XW1⊙σ(XW2))W3

[0039] Where Output represents the output of the Gated MLP module; X is the input feature; W1, W2, and W3 are learnable weight matrices. W1 maps the input feature X to a linear transformation space, and W2 generates the gating signal gating. signal, which controls the activation degree of the feature; W3 performs the final linear transformation on the fused features to generate the output; ⊙ is element-by-element multiplication; σ is the activation function; XW1 is the main feature path, which means that the feature representation of the input feature X after the linear transformation W1 retains the main information flow; σ(XW2) is the gating signal, which means that the gating signal generated after the input feature X passes through the linear transformation W2 and the activation function σ, which is used to control which parts of the main feature path need to be retained or suppressed; (XW1⊙σ(XW2)) is the gated feature fusion, which means that the main feature path XW1 and the gating signal σ(XW2) are combined by element-by-element multiplication to achieve selective enhancement or suppression of the feature; (XW1⊙σ(XW2))W3 is the final linear transformation, which means that the result of the gated feature fusion is passed through the linear transformation W3 again to generate the final output feature.

[0040] Furthermore, in step 5, L1 loss and mean square error (MSE) are used in combination with physical constraints to optimize model parameters. The calculation formula is as follows:

[0041]

[0042] Where, is the total loss value; h pred is the predicted thickness; h true is the true thickness; |h pred -h true | is the absolute error, i.e. L1 loss; (h pred -h true ) 2 is the square error, i.e. MSE loss; α is the weight coefficient of L1 loss; β is the weight coefficient of MSE loss; ∑ is the summation operation;

[0043] Physical constraints introduce the scattering physical characteristics of millimeter-wave SAR into the loss function to enhance the physical interpretability of the model output; self-supervised learning uses unlabeled data to train the model, extracts common features through data enhancement or contrastive learning methods, and reduces dependence on large-scale labeled data.

[0044] Furthermore, in step 7, the UAV is equipped with a millimeter-wave SAR sensor, which combines the IMU and GNSS to synchronously collect attitude and positioning data, and flies along the power transmission line to obtain multi-polarization and multi-time series radar data. The collection process supports local preliminary pre-processing and transmits high-resolution raw data in real time to the ground computing node via a wireless link.

[0045] Edge computing devices with GPU acceleration are deployed on the ground side, hosting a deep neural network structure and responsible for performing inference tasks including data enhancement, feature extraction, multimodal fusion, and thickness inversion. The ice thickness map and corresponding confidence distribution output by the model serve as the core decision-making basis to determine whether the current line ice coverage exceeds the safety threshold.

[0046] The power grid monitoring system synchronously uploads the inference results to the grid-side intelligent monitoring platform through a secure communication protocol; the monitoring system integrates and analyzes the prediction results with historical operating data, meteorological models, etc., sets multi-level warning thresholds, and makes joint judgments on the predicted thickness and confidence intervals; when the system determines that the line has a moderate to severe icing risk, it automatically triggers the alarm mechanism, and links the dispatching system to generate maintenance tasks and push them to relevant operation and maintenance terminals.

[0047] The beneficial effects of the present invention are:

[0048] 1. Enhanced feature modeling capabilities: Using the Swin Transformer V2 as the backbone structure, combined with the GatedMLP module, significantly improves the ability to extract complex textures and spatial relationships in millimeter-wave SAR images, breaking through the limitations of traditional convolutional networks and enhancing the ability to model long-range dependencies.

[0049] 2. Deep cross-modal information fusion: The Cross-Modal Transformer is used to achieve semantic alignment and fusion of SAR imagery with meteorological and line operation parameters. This surpasses the fusion method of traditional two-stream networks, excels in joint modeling of multi-source heterogeneous data, and significantly improves the model's adaptability to complex environmental changes.

[0050] 3. Physical constraints enhance credibility: A physical consistency decoder branch is introduced, and backscattering physical constraints are added to the loss function. This effectively improves the physical interpretability of the model output, suppresses the deviant behavior of purely data-driven models, and enhances credibility and acceptability in engineering deployment.

[0051] 4. Uncertainty estimation supports risk-controlled reasoning: An integrated uncertainty estimation channel provides confidence assessment for each ice thickness prediction value, supports confidence interval judgment and risk warning, and provides safer and more reliable support for power grid operation and maintenance decision-making.

[0052] 5. Generalization capability under low-label sample conditions: Combining self-supervised learning with physics-aware GAN technology, it uses unlabeled or small amounts of labeled data to improve model generalization performance, alleviating the practical problems of high data acquisition costs and scarce annotations, and significantly reducing training data requirements.

[0053] 6. Real-time and edge deployment capabilities: Using a lightweight Transformer decoder and dual-path architecture to optimize inference speed, the model can run efficiently on GPU-accelerated servers or edge devices. Under standard hardware conditions, the prediction time is controlled within 0.5 seconds, meeting the needs of industrial-grade real-time applications.

[0054] 7. Significantly enhanced anti-interference and robustness: Under complex meteorological conditions such as strong noise, high humidity, low temperature, and snowfall, the algorithm's thickness inversion error is less than 5%, improving accuracy by 20%-35% compared to traditional methods. Even in low signal-to-noise ratio environments, the accuracy remains above 85%, demonstrating excellent anti-interference capabilities.

[0055] 8. Significantly improve power grid security and economic benefits: This algorithm has the advantages of high precision, strong robustness, and low deployment threshold. It can be seamlessly integrated into existing drone inspection and power grid intelligent monitoring systems, significantly improving icing disaster warning capabilities, reducing manual inspection costs and equipment losses, and has extremely high engineering application value and promotion prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction to the drawings required for use in the embodiments will be given below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0057] Figure 1 This is a flow chart of the millimeter wave SAR ice thickness inversion algorithm based on deep learning of the present invention;

[0058] Figure 2 This is a schematic diagram of the millimeter wave SAR ice thickness inversion algorithm of the present invention;

[0059] Figure 3 This is the deep learning network structure diagram for millimeter wave SAR ice thickness inversion in the present invention. DETAILED DESCRIPTION

[0060] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0061] To address the issues of insufficient accuracy and poor real-time performance of ice monitoring under complex meteorological conditions, this embodiment provides a deep learning-based millimeter-wave SAR transmission line ice thickness inversion algorithm. This algorithm achieves high-precision inversion and intelligent early warning of transmission line ice thickness through millimeter-wave SAR data acquisition, data preprocessing, deep learning modeling, multimodal data fusion, model training optimization, and real-time inference. This method can provide efficient and accurate monitoring under complex meteorological conditions, significantly improving the intelligent level of safe power grid operation.

[0062] To achieve UAV data collection, wireless transmission, deep learning model training and inference on ground high-performance computing servers, and seamless integration with power grid monitoring systems, real-time monitoring and intelligent early warning support are provided. The deep learning-based millimeter wave SAR transmission line ice thickness inversion is based on the following system architecture:

[0063] The drone is responsible for collecting millimeter-wave SAR data and transmitting it to the ground station via wireless communication. The ground station is equipped with high-performance computing servers and runs deep learning models for data preprocessing, feature extraction, thickness inversion, and early warning analysis. The power grid monitoring system receives ice thickness forecasts, combines them with historical data for comprehensive analysis, and triggers intelligent early warning mechanisms.

[0064] The dynamic operation process is as follows: drones collect data and transmit it in real time; the ground station preprocesses the data and performs deep learning inference; the prediction results are fed back to the power grid monitoring system to trigger early warning or dispatch maintenance.

[0065] like Figure 1 and Figure 2 As shown in Figure 1, the deep learning-based millimeter wave SAR transmission line ice thickness inversion algorithm includes the following steps:

[0066] Step 1 Data Collection:

[0067] Equipment configuration: A drone equipped with a millimeter-wave synthetic aperture radar (SAR) sensor operates in the 30-300 GHz frequency band. This frequency band offers high resolution and strong penetration, enabling the acquisition of clear backscattered images in complex weather conditions such as rain, snow, and fog.

[0068] The drone is equipped with a high-precision inertial measurement unit (IMU) and a global navigation satellite system (GNSS), which are used to record the drone's position, attitude, and speed information in real time, providing an accurate reference for subsequent data correction.

[0069] Data Acquisition: A UAV equipped with a millimeter-wave SAR sensor conducts side-view imaging along power transmission lines, acquiring multi-view, multi-time series millimeter-wave SAR echo data. Multi-polarization modes (VV, HH, VH, and HV) are used to acquire backscattered signals in different polarization states, improving the ability to distinguish ice-covered targets. Combined with interferometric SAR (InSAR) or polarimetric SAR (PolSAR), coherence analysis enhances target information and effectively suppresses noise interference.

[0070] Data characteristics: The collected raw SAR data has a resolution of 256×256 and contains three channels of information, such as complex echo amplitude and phase. The data is transmitted via wireless communication to a ground-based processing system comprised of high-performance computing servers with built-in GPU acceleration for deep learning model training and inference.

[0071] This step uses a millimeter-wave SAR sensor mounted on a drone to acquire multi-polarization, multi-time series radar echo data from the transmission line. This data is then combined with high-precision IMU and GNSS to record the drone's position, attitude, and motion. Interferometric SAR or polarimetric SAR techniques are also used to enhance target information and suppress noise interference, providing high-quality input data for subsequent data preprocessing and deep learning modeling.

[0072] Step 2: Data preprocessing:

[0073] Millimeter-wave SAR data is affected by factors such as antenna beam directivity, platform attitude variations, and system noise, requiring a series of preprocessing steps to ensure data quality and stability. This preprocessing includes slant range correction, polarimetric denoising, backscatter feature extraction, and data enhancement, providing high-quality input for subsequent deep learning feature extraction.

[0074] The specific steps are as follows:

[0075] like Figure 1 and Figure 2 As shown, slant range correction is performed first. Because the SAR system uses side-view imaging, the original image is geometrically distorted, requiring slant range projection correction. The image is converted from slant range coordinates to a ground coordinate system to eliminate geometric distortion. Motion compensation is performed using high-precision IMU and GNSS data to correct errors caused by the UAV platform's motion.

[0076] Polarimetric denoising is then performed: Polarimetric coherence filtering is used to remove speckle noise. This improves the stability of the target's scattering characteristics and ensures that data quality meets the expected standards. An adaptive filtering algorithm is also used to further suppress noise interference.

[0077] Backscattering characteristics are then extracted: the Radar Cross Section (RCS) backscatter coefficients are calculated for different polarization channels to extract the scattering characteristics of the target area. Key characteristic parameters include polarization ratio, entropy, and anisotropy. These features preserve key information about the target area and provide foundational support for subsequent deep learning modeling.

[0078] Finally, data augmentation is performed: a generative adversarial network (GAN) is used to generate additional SAR samples to expand the training dataset. This data augmentation method effectively compensates for the lack of real annotated data and improves the model's generalization ability.

[0079] This step performs slant range correction on the raw SAR data to eliminate geometric distortion, polarization denoising to remove coherent speckle noise, backscattering characteristic extraction to calculate polarization ratio, entropy and other features, and data enhancement using GAN to generate additional samples to ensure the quality and stability of the input data, laying a solid foundation for subsequent deep learning modeling.

[0080] Step 3: Feature extraction and modeling:

[0081] like Figure 2 and Figure 3 As shown in the figure, the millimeter-wave SAR ice thickness inversion network in this embodiment adopts a multi-branch, highly complex deep neural structure, combined with the Swin Transformer V2 backbone structure, Gated MLP module, cross-modal attention fusion, multi-path decoder and physical consistency constraint module to build a thickness inversion model that is highly adapted to the millimeter-wave SAR imaging characteristics and multi-modal input.

[0082] Input layer and patch embedding: The input data consists of a 256×256×3, three-channel millimeter-wave SAR image containing multi-polarization or multi-time series information, as well as synchronized meteorological parameters and power line operation data, i.e., a multimodal numerical vector. The image data is encoded into low-dimensional feature blocks through the patch embedding module, such as a 7×7 convolution followed by strided projection, for subsequent processing by the Transformer module.

[0083] The backbone feature extraction module uses a two-level Swin Transformer V2 encoding block to extract hierarchical image features. Each level contains the following components:

[0084] Window-based Multi-head Self-Attention (W-MSA): Calculates attention in a local window; LayerScale module: Adjusts the stability of the residual path; Shifted Window strategy: Enhances information transfer between regions; Residual connection and normalization operation: Improves training stability and representation capabilities.

[0085] After the second layer of encoding, the Adaptive Spatial Attention and SE channel attention modules are added to further enhance the local target perception capability, which is especially suitable for the detection of long strip targets in the power line area.

[0086] The deep reasoning module introduces the ST-Transformer structure to model cross-frame feature correlations in time series or polarization domains, improving the modeling capabilities of dynamic thickness evolution and polarization characteristic changes.

[0087] The decoder adopts a dual-path upsampling architecture; the model uses two parallel decoding paths: Path 1: a lightweight convolutional decoder with layer-by-layer upsampling and skip connections; Path 2: a Transformer-based attention decoder that enhances semantic information reconstruction. The two decoding paths are finally fused at the intermediate layer and fed into the output module.

[0088] The multi-head output structure includes: Head1, Head2, Head3;

[0089] Head1 is thickness prediction, and the output tensor size is H×W, representing the pixel-level ice thickness;

[0090] Head2 is the uncertainty estimate, output Represents the confidence interval bounds for thickness prediction, modeled using MCDropout or Bayesian estimation;

[0091] Head3 is a physical consistency decoder that feeds the thickness prediction result into the electromagnetic scattering affine layer to calculate the residual loss between the theoretical backscattering model:

[0092]

[0093] Where σ(x) is the uncertainty estimate of the model output; is the spatial dimension of the output, H and W represent the height and width of the output feature map, respectively, corresponding to the spatial resolution of the millimeter-wave SAR image; x is the input data, i.e., the millimeter-wave SAR image; is the physical consistency loss; N is the total number of samples or pixels; σ pred (h i ) is the backscattering coefficient predicted by the model; σ theory (hi ) is the theoretical backscattering coefficient; h i Ice thickness of the i-th sample; (σ pred (h i )-σ theory (h i )) 2 is the square of the physical consistency error of a single sample.

[0094] This structure is significantly superior to the traditional CNN single-path structure, taking into account feature expression, spatial modeling capabilities, modal fusion capabilities and reasoning credibility. It is particularly suitable for complex task scenarios of millimeter-wave SAR data inversion and can effectively improve the thickness estimation performance in low signal-to-noise ratio and high uncertainty environments.

[0095] Step 4: Multimodal data fusion:

[0096] like Figure 1 and Figure 2 As shown in the figure, millimeter-wave SAR image data is fused with meteorological data and transmission line operating parameters. The meteorological data includes temperature, humidity, and wind speed, and the transmission line operating parameters include current and voltage. Feature-level and decision-level fusion strategies are adopted to improve the adaptability and robustness of the model to complex environments.

[0097] The feature-level fusion process involves mapping meteorological data and operating parameters through independent MLPs and then inputting them into Cross-Modal Attention as key-value pairs to align with image features:

[0098] F fusion =Attention(Q=F img W q ,K=F env W k ,V=F env W v )

[0099] The fused output is concatenated with the image backbone features for subsequent high-level reasoning;

[0100] Where, F fusion is the fused feature representation; Attention is the attention mechanism function; Q = F img W q In this example, Q is the query matrix, F img is the image feature; W q is the query projection matrix; K = F env W k In this example, K is the bond matrix, F env is the environmental data feature, W k is the key projection matrix; V = F env In W, V is the value matrix, Wv is the value projection matrix.

[0101] The decision-level fusion process involves training two independent neural networks, one for SAR data and the other for multimodal data, and fusing the final prediction results through weighted averaging.

[0102] At the same time, the Gated MLP module is introduced to replace part of the convolution structure. Its basic unit is:

[0103] Output=(XW1⊙σ(XW2))W3

[0104] Where Output represents the output of the Gated MLP module; X is the input feature; W1, W2, and W3 are learnable weight matrices. W1 maps the input feature X to a linear transformation space, and W2 generates the gating signal gating. signal, which controls the activation degree of the feature; W3 performs the final linear transformation on the fused features to generate the output; ⊙ is element-by-element multiplication; σ is the activation function; XW1 is the main feature path, which means that the feature representation of the input feature X after the linear transformation W1 retains the main information flow; σ(XW2) is the gating signal, which means that the gating signal generated after the input feature X passes through the linear transformation W2 and the activation function σ, which is used to control which parts of the main feature path need to be retained or suppressed; (XW1⊙σ(XW2)) is the gated feature fusion, which means that the main feature path XW1 and the gating signal σ(XW2) are combined by element-by-element multiplication to achieve selective enhancement or suppression of the feature; (XW1⊙σ(XW2))W3 is the final linear transformation, which means that the result of the gated feature fusion is passed through the linear transformation W3 again to generate the final output feature.

[0105] This structure effectively models the spatial nonlinear relationship under a wide receptive field, and completes the splicing and normalization of features under different receptive fields through the multi-scale fusion module MSFF.

[0106] Step 5: Model training and optimization:

[0107] like Figure 1 and Figure 2 As shown in the figure, this step optimizes the model parameters by combining L1 loss and mean square error (MSE) loss, and introduces physical constraints and self-supervised learning methods to improve the accuracy, generalization ability and physical interpretability of the ice thickness inversion model.

[0108] Loss function design:

[0109] The objective function adopts a combination of L1 loss and mean square error MSE loss, and the calculation formula is as follows:

[0110]

[0111] Where, is the total loss value, which represents the error between the model prediction value and the true value. It is used to measure the performance of the model. The smaller the loss value, the closer the model prediction result is to the true value. pred is the predicted thickness, which is the ice thickness value predicted by the model based on the input data; h true is the true thickness, which is the true thickness of ice covering the transmission line obtained through manual measurement or auxiliary instruments and is used as the annotation data for training the model; pred -h true | is the absolute error, i.e. L1 loss, which represents the absolute difference between the predicted value and the true value. L1 loss is less sensitive to outliers and can better handle noisy data; (h pred -h true ) 2 is the square error, namely MSE loss, which represents the square of the difference between the predicted value and the true value. MSE loss is more sensitive to larger errors and can more accurately punish larger prediction deviations; α is the weight coefficient of L1 loss, which is used to adjust the importance of L1 loss in the total loss. The larger α is, the more the model tends to optimize the absolute error; β is the weight coefficient of MSE loss, which is used to adjust the importance of MSE loss in the total loss. The larger β is, the more the model tends to optimize the square error; ∑ is the summation operation.

[0112] L1 loss is insensitive to outliers and can reduce the impact of noisy data; MSE loss is more sensitive to larger errors and can more strictly penalize larger prediction deviations. By adjusting the values of α and β, you can find the optimal balance between the two losses.

[0113] Physical constraints: Incorporating the scattering physics of millimeter-wave SAR into the loss function, such as the relationship between the backscatter coefficient and ice thickness, enhances the physical interpretability of the model output. For example, the theoretical relationship between ice thickness and backscattering characteristics can be incorporated into the loss function as a constraint to ensure that the model predictions conform to physical laws. Introducing physical constraints can improve the correlation between model output and actual physical properties, suppress overfitting, and enhance the model's robustness in complex environments.

[0114] Self-supervised learning: Use unlabeled data to train the model and extract common features through the following methods:

[0115] Generate additional samples using generative adversarial networks (GANs) or image transformation techniques; learn the underlying representation of the data by comparing positive and negative sample pairs. Self-supervised learning reduces the reliance on large amounts of labeled data and improves the model's generalization ability, enabling it to perform better on unseen data.

[0116] Training Process: The training dataset consists of field-collected millimeter-wave SAR images and annotated ice thickness data obtained through manual or instrumental measurements. Data augmentation techniques are used to expand the sample size and improve the model's generalization capabilities. Backpropagation and gradient descent algorithms are used to update network parameters, gradually minimizing the loss function. Self-supervised learning methods are introduced to enable the model to adaptively extract useful features from unlabeled data. SAR scattering physics principles are incorporated to mitigate overfitting and enhance the model's physical consistency and credibility.

[0117] This step combines L1 and MSE losses and introduces physical constraints to ensure accurate and physically interpretable model predictions. It also extracts common features from unlabeled data, reducing reliance on large amounts of labeled data. Furthermore, it employs data augmentation techniques and a gradient descent algorithm to improve the model's generalization and robustness. This approach significantly improves the accuracy and adaptability of the ice thickness inversion model, providing reliable technical support for transmission line monitoring under complex meteorological conditions.

[0118] Step 6 Thickness inversion and real-time inference:

[0119] like Figure 1 and Figure 2 As shown in the figure, based on the trained deep learning model, real-time collected millimeter-wave SAR images and multimodal data, such as meteorological data and line operation parameters, are input to predict the ice thickness of transmission lines, and the early warning mechanism is triggered in combination with confidence analysis.

[0120] The real-time inference process is as follows: A millimeter-wave SAR image is input, and a convolutional neural network (CNN) is used to extract local texture features such as edges and structure. The Transformer architecture is used to capture long-range spatial dependencies in the image and optimize the weight distribution of spatial features. Meteorological data such as temperature, humidity, and wind speed, as well as transmission line operating parameters such as current and voltage, are integrated to further refine the ice thickness prediction results. The predicted ice thickness is output, and the reliability of the prediction is determined through confidence analysis. When the predicted ice thickness reaches or exceeds a preset safety threshold, the system automatically triggers an alarm, notifying operations and maintenance personnel to take action.

[0121] Early warning mechanism: When the predicted ice thickness exceeds the preset safety threshold, the system automatically issues an alarm and dispatches on-site maintenance personnel to conduct inspections or take de-icing measures to ensure the safe operation of the power grid.

[0122] Step 7: System integration and dynamic operation:

[0123] In engineering practice, the millimeter-wave SAR ice thickness inversion algorithm of the present invention is a complete system composed of three parts: the UAV end, the edge computing node and the power grid monitoring platform. It has high integration and real-time response capabilities and can adapt to power grid inspection scenarios in complex terrain and meteorological environments. The UAV end is equipped with a millimeter-wave SAR sensor, which combines IMU and GNSS for synchronous attitude and positioning acquisition, and flies along the transmission line to obtain multi-polarization and multi-time series radar data. During the acquisition process, the system supports the completion of preliminary pre-processing operations locally, such as slant range correction and platform motion compensation, and transmits high-resolution raw data in real time to the ground computing node through wireless links such as 5G or millimeter-wave communication.

[0124] Edge computing devices with GPU acceleration capabilities are deployed on the ground side, carrying the deep neural network structure proposed in this invention and responsible for performing reasoning tasks including data enhancement, feature extraction, multimodal fusion, and thickness inversion. The ice thickness map and corresponding confidence distribution output by the model will serve as the core decision-making basis for further determining whether the current line ice coverage exceeds the safety threshold. This calculation process supports parallel processing of batch data and automatic cache updates, and can achieve end-to-end real-time response without relying on a central server, significantly improving inspection frequency and system reliability.

[0125] The inference results are synchronously uploaded to the grid-side intelligent monitoring platform via a secure communication protocol. The monitoring system integrates and analyzes the prediction results with historical operating data and meteorological models, sets multi-level warning thresholds, and makes joint judgments on the predicted thickness and confidence intervals. When the system determines that the line is at risk of moderate to severe icing, it automatically triggers the alarm mechanism and links the dispatching system to generate maintenance tasks, which are pushed to the relevant operation and maintenance terminals. The entire process, from data collection to warning triggering, can be completed in seconds. It has the characteristics of high timeliness, high accuracy, and high adaptability, and can be widely used in tasks such as routine inspections of transmission lines, disaster monitoring, and emergency response.

Claims

1. A millimeter-wave SAR ice thickness inversion algorithm based on deep learning, characterized by: The deep learning-based millimeter-wave SAR ice thickness inversion algorithm includes the following steps: Step 1 Data Collection: Use the millimeter-wave SAR sensor on the drone to obtain multi-polarization VV, HH, VH, HV, and multi-time series radar echo data of the transmission line. Combined with the high-precision inertial measurement unit (IMU) and global navigation satellite system (GNSS), the drone's position, attitude, and motion information are recorded. Step 2: Data preprocessing: The raw SAR data collected by the drone is subjected to slant range correction to eliminate geometric distortion, polarization denoising to remove coherent speckle noise, backscattering characteristics extraction to calculate polarization ratio and entropy, and data enhancement using a generative adversarial network (GAN) to ensure input data quality and stability. Step 3: Feature extraction and modeling: Using the Swin Transformer V2 backbone structure combined with Gated MLP, cross-modal attention fusion, and a dual-path decoder, a multi-branch deep neural network is constructed to extract local and global features of millimeter-wave SAR images. Step 4: Multimodal data fusion: Integrate millimeter-wave SAR images, meteorological data, and line operation parameters through feature-level fusion (Cross-Modal Attention) and decision-level fusion. Step 5: Model training and optimization: L1 loss and mean square error (MSE) are used in conjunction with physical constraints to optimize model parameters, and self-supervised learning methods are introduced to improve the model's generalization ability for unlabeled data. Step 6: Thickness inversion and real-time inference: Based on the trained deep learning model, real-time millimeter-wave SAR images and multimodal data are input to predict ice thickness on transmission lines. Confidence analysis is then used to trigger an early warning mechanism. Step 7: System Integration and Dynamic Operation: The drone collects millimeter-wave SAR data and transmits it wirelessly to the edge computing node. GPU acceleration is used to complete deep learning inference. The results are uploaded to the power grid monitoring platform to trigger intelligent early warnings, achieving end-to-end real-time ice cover monitoring and emergency response.

2. The deep learning-based millimeter wave SAR ice thickness inversion algorithm according to claim 1 is characterized in that: This deep learning-based millimeter-wave SAR ice thickness inversion algorithm is based on the following system architecture: the drone end is responsible for millimeter-wave SAR data collection and transmits the data to the ground station via wireless communication; the ground station is equipped with a high-performance computing server that runs a deep learning model for data preprocessing, feature extraction, thickness inversion and early warning analysis; the power grid monitoring system receives the ice thickness prediction results, combines them with historical data for comprehensive analysis, and triggers the intelligent early warning mechanism.

3. The deep learning-based millimeter wave SAR ice thickness inversion algorithm according to claim 1 is characterized in that: In step 1, a UAV equipped with a millimeter-wave SAR sensor performs side-view imaging along the transmission line to obtain multi-view and multi-time series millimeter-wave SAR echo data; multi-polarization modes VV, HH, VH, and HV are used to obtain backscattered signals under different polarization states; combined with interferometric SAR or polarimetric SAR, target information is enhanced and noise interference is suppressed.

4. The deep learning-based millimeter wave SAR ice thickness inversion algorithm according to claim 1 is characterized in that: In step 2, slant range correction is to correct the geometric distortion of the original SAR image and convert the image from the slant range coordinate system to the ground coordinate system; polarimetric coherence filtering is used to remove coherent speckle noise and improve the stability of the target scattering characteristics; Backscattering characteristic extraction is to calculate the backscattering coefficient RCS under different polarization channels and extract the scattering characteristics of the target, including polarization ratio, entropy, and anisotropy.

5. The deep learning-based millimeter wave SAR ice thickness inversion algorithm according to claim 1 is characterized in that: In step 3, the millimeter-wave SAR ice thickness inversion network adopts a multi-branch, highly complex deep neural structure, combined with the SwinTransformer V2 backbone structure, Gated MLP module, cross-modal attention fusion, multi-path decoder and physical consistency constraint module to build a thickness inversion model that is highly adapted to millimeter-wave SAR imaging characteristics and multi-modal input; The input data includes three-channel millimeter-wave SAR images and synchronized meteorological parameters and transmission line operation data. The image data is encoded into low-dimensional feature blocks through the Patch Embedding module for subsequent processing by the Transformer module. The backbone feature extraction module uses a two-stage Swin Transformer V2 encoding block to extract hierarchical image features. Each stage contains the following components: Window-based Multi-head Self-Attention to calculate attention in a local window; LayerScale module to adjust the residual path stability; Shifted Window strategy to enhance information transfer between regions; residual connection and normalization operations; The deep reasoning module introduces the ST-Transformer structure to model cross-frame feature correlations in time series or polarization domains; The decoder has a dual-path upsampling structure; the model uses two parallel decoding paths: Path 1 is a lightweight convolutional decoder with layer-by-layer upsampling and skip connections; Path 2 is a Transformer-based attention decoder; The two decoding paths are finally fused in the middle layer and fed into the output module; The multi-head output structure includes: Head1, Head2, Head3; Head1 is thickness prediction, and the output tensor size is H×W, representing the pixel-level ice thickness; Head2 is the uncertainty estimate, output Represents the confidence interval bounds for thickness prediction, modeled using MCDropout or Bayesian estimation; Head3 is a physical consistency decoder that feeds the thickness prediction result into the electromagnetic scattering affine layer to calculate the residual loss between the theoretical backscattering model: Where σ(x) is the uncertainty estimate of the model output; is the spatial dimension of the output, H and W represent the height and width of the output feature map, respectively, corresponding to the spatial resolution of the millimeter-wave SAR image; x is the input data, i.e., the millimeter-wave SAR image; is the physical consistency loss; N is the total number of samples or pixels; σ pred (h i ) is the backscattering coefficient predicted by the model; σ theory (h i ) is the theoretical backscattering coefficient; h i Ice thickness of the i-th sample; (σ pred (h i )-σ theory (h i )) 2 is the square of the physical consistency error of a single sample.

6. The deep learning-based millimeter wave SAR ice thickness inversion algorithm according to claim 1 is characterized in that: In step 4, the feature-level fusion process involves mapping meteorological data and operating parameters through independent MLPs and then inputting them into Cross-Modal Attention as key-value pairs to align with image features: F fusion =Attention(Q=F img W q ,K=F env W k ,V=F env W v ) The fused output is concatenated with the image backbone features for subsequent high-level reasoning; Where, F fusion is the fused feature representation; Attention is the attention mechanism function; Q = F img W q In this example, Q is the query matrix, F img is the image feature; W q is the query projection matrix; K = F env W k In this example, K is the bond matrix, F env is the environmental data feature, W k is the key projection matrix; V = F env In W, V is the value matrix, W v is the value projection matrix.

7. The deep learning-based millimeter wave SAR ice thickness inversion algorithm according to claim 1, characterized in that: In step 4, the decision-level fusion process is to train two independent neural networks, one for SAR data and the other for multimodal data, and fuse the final prediction results through weighted averaging; At the same time, the Gated MLP module is introduced to replace part of the convolution structure. Its basic unit is: Output=(XW1⊙σ(XW2))W3 Where, Output represents the output of the Gated MLP module; X is the input feature; W1, W2, and W3 are learnable weight matrices. W1 maps the input feature X to a linear transformation space, and W2 generates a gating signal to control the activation degree of the feature. W3 performs the final linear transformation on the fused features to generate the output; ⊙ is element-by-element multiplication; σ is the activation function; XW1 is the main feature path, which represents the feature representation of the input feature X after the linear transformation W1, retaining the main information flow; σ(XW2) is the gating signal, which represents the gating signal generated after the input feature X passes through the linear transformation W2 and the activation function σ, and is used to control which parts of the main feature path need to be retained or suppressed; (XW1⊙σ(XW2)) is the gated feature fusion, which means that the main feature path XW1 and the gating signal σ(XW2) are combined by element-by-element multiplication to achieve selective enhancement or suppression of features; (XW1⊙σ(XW2))W3 is the final linear transformation, which means that the result of the gated feature fusion is passed through the linear transformation W3 again to generate the final output feature.

8. The deep learning-based millimeter wave SAR ice thickness inversion algorithm according to claim 1, characterized in that: In step 5, L1 loss and mean square error (MSE) are used in combination with physical constraints to optimize model parameters. The calculation formula is as follows: Where, is the total loss value; h pred is the predicted thickness; h true is the true thickness; |h pred -h true | is the absolute error, i.e. L1 loss; (h pred -h true ) 2 is the square error, i.e. MSE loss; α is the weight coefficient of L1 loss; β is the weight coefficient of MSE loss; Σ is the summation operation; Physical constraints introduce the scattering physical characteristics of millimeter-wave SAR into the loss function to enhance the physical interpretability of the model output; self-supervised learning uses unlabeled data to train the model, extracts common features through data enhancement or contrastive learning methods, and reduces dependence on large-scale labeled data.

9. The deep learning-based millimeter wave SAR ice thickness inversion algorithm according to claim 1, characterized in that: In step 7, the UAV is equipped with a millimeter-wave SAR sensor, which combines the IMU and GNSS to simultaneously collect attitude and positioning data, and flies along the power transmission line to obtain multi-polarization and multi-time series radar data. The acquisition process supports local preliminary pre-processing and transmits high-resolution raw data in real time to the ground computing node via a wireless link. Edge computing devices with GPU acceleration capabilities are deployed on the ground side to carry deep neural network structures and perform reasoning tasks including data enhancement, feature extraction, multimodal fusion, and thickness inversion. The ice thickness map and corresponding confidence distribution output by the model serve as the core decision-making basis to determine whether the current line ice coverage exceeds the safety threshold; The power grid monitoring system synchronously uploads the inference results to the grid-side intelligent monitoring platform through a secure communication protocol; The monitoring system integrates and analyzes the prediction results with historical operating data, meteorological models, etc., sets multi-level warning thresholds, and makes joint judgments on the predicted thickness and confidence intervals; when the system determines that there is a risk of moderate to severe icing on the line, it automatically triggers the alarm mechanism, and links the dispatching system to generate maintenance tasks and push them to relevant operation and maintenance terminals.

Images