An ice detection method based on multi-source data feature interaction

Abstract

The present invention discloses an ice detection method based on the interaction of multi-source data features, which belongs to the field of image processing technology. The method comprises the following steps: inputting an original ice image into a dynamic routing image enhancement autoencoder to generate a high-quality enhanced image; extracting multi-level features of the image, including local ice features, global ice features and meteorological time series features; after uniform scale mapping and feature precoding, the local ice features, global ice features and meteorological time series features are interactively fused through a bidirectional cross-modal attention mechanism and feature splicing operations to output ice fusion features; inputting the ice fusion features into a multi-task prediction module, extracting deep features through a residual fusion encoder and completing task mapping, and outputting the corresponding ice type and ice thickness level; the present invention realizes efficient ice detection through the fusion of meteorological time series information and features, can adapt to different climate and terrain conditions, and improve the safety and stability of transmission line operation.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to an ice cover detection method based on multi-source data feature interaction. Background Art

[0002] Identifying ice on transmission lines is a key technology in power systems, particularly in cold regions. Icing is a frequent occurrence, increasing the weight of lines and exacerbating conductor galloping, leading to breakage or equipment damage, and thus compromising the safety and stability of power systems. Therefore, timely and accurate identification of ice is crucial. Traditional manual inspections rely on on-site inspections. While intuitive and reliable, they suffer from inefficiencies, missed inspections, and delays due to the extensive nature of transmission lines and complex environments. In recent years, automated monitoring technologies have become a research hotspot, primarily employing drones, remote sensing, LiDAR, and infrared imaging. These technologies utilize image processing, machine learning, and deep learning algorithms to automatically identify and analyze ice. While these technologies have made some progress, they still face challenges. Complex weather conditions such as snow, fog, and backlighting can affect image clarity and reduce recognition accuracy. Furthermore, the high cost of sample annotation and data scarcity limit deep learning model training. Furthermore, engineering sites require high computing resources and high response speeds, placing higher demands on lightweight and real-time models. How to efficiently and accurately identify ice cover in massive amounts of data remains a difficult point in technological development. Therefore, improving recognition accuracy, reducing false detections and missed detections, and improving real-time performance are key issues that need to be urgently addressed in current ice cover recognition technology.

[0003] Image-based ice detection methods install image acquisition equipment on towers or other monitoring facilities to capture and analyze ice images to determine the ice status of transmission lines. Common methods include those based on visual sensors and K-SVD denoising, which can effectively remove image noise and improve recognition accuracy. However, these methods are computationally complex and highly dependent on lighting conditions, affecting real-time performance. Another approach uses drones equipped with binocular cameras to capture multi-view images and combine deep learning with feature matching to measure ice thickness. However, drones are significantly affected by inclement weather and consume high computing resources, making them difficult to achieve real-time monitoring around the clock. Furthermore, methods based on Canny edge detection and Hough transform can extract ice contours, but are sensitive to background interference and easily affected by ambient lighting and occlusion.

[0004] In deep learning-based transmission line ice detection, a dynamic routing image enhancement autoencoder is used to process raw ice images. This effectively enhances dark areas and improves contrast, avoiding detail loss, artifacts, or over-processing. Furthermore, meteorological data is used to extract meteorological time-series features, providing more discriminative meteorological semantic information for ice detection, effectively improving the model's ability to perceive ice conditions in complex environments. Most image-based ice detection methods often overlook the influence of meteorological and topographical factors, which play a significant role in ice formation and thickness distribution. Meteorological conditions such as temperature, humidity, and wind speed in different regions, as well as the terrain on which transmission lines are located, significantly influence the rate of ice accumulation and distribution. Therefore, combining deep learning models with meteorological and topographical data can more accurately predict ice conditions and optimize detection results. Deep learning convolutional neural networks not only effectively identify ice features in images but also analyze the impact of environmental factors on ice accumulation by combining multi-source data, thereby improving detection accuracy. Leveraging deep learning technology and integrating external environmental information will significantly improve the performance of transmission line ice detection. Summary of the Invention

[0005] In response to the shortcomings of the background technology, the present invention provides a method for detecting icing on transmission lines, including the joint identification of icing type and icing thickness level, and the use of meteorological time series information and feature fusion to improve detection accuracy. At the same time, the icing detection method is optimized for complex meteorological conditions, terrain changes and low visibility environments, so that it can stably and efficiently complete icing detection on transmission lines in different environments.

[0006] The present invention adopts the following technical solutions:

[0007] An ice detection method based on multi-source data feature interaction specifically includes the following steps:

[0008] S1. Collect the actual original ice-covered images of the transmission lines through monitoring cameras on the transmission lines, organize the images into image sequences in chronological order, and simultaneously obtain DEM data and meteorological data;

[0009] S2, input the normalized ice-covered image into the dynamic routing image enhancement autoencoder and output the enhanced image;

[0010] S3. The enhanced image and DEM data are input into the Terrafuse-BiNet dual-branch feature extraction network in parallel. The deformable multi-scale detail feature extraction branch extracts edge textures, and the height difference guided attention perception branch integrates terrain for spatial modeling. Finally, local ice cover features and global ice cover features are obtained respectively.

[0011] S4. Input meteorological data into the TRAM-Net meteorological feature extraction module, use the local meteorological variation rate to guide the attention bias, complete feature encoding and output meteorological time series features;

[0012] S5: After unified scale mapping and feature precoding, the local and global features of ice cover and meteorological time series features are interactively fused through a bidirectional cross-modal attention mechanism and feature splicing operations, and finally the scale-aligned multi-modal ice cover fusion features are output;

[0013] S6. The ice cover fusion features are input into the multi-task prediction module, where the residual fusion encoder extracts deep features, and then the task-aware multi-head self-attention module extracts type features and thickness level features, and finally inputs them into the lightweight classifier to obtain the corresponding ice cover type and ice cover thickness level.

[0014] As a further preferred embodiment of the ice detection method based on multi-source data feature interaction of the present invention, in step S2, obtaining an enhanced image specifically includes the following steps:

[0015] S2.1、 Original ice cover image at the moment Input the noise type discriminator branch of the dynamic routing image enhancement autoencoder. After passing through the convolution layer, global average pooling layer and a fully connected layer within the branch, the softmax function outputs the category label with the largest probability value. , Indicates that the image contains obvious Gaussian noise. Indicates that the image is blurry or has low contrast. Indicates that the image does not have too much noise and only needs light enhancement. The calculation process is expressed as:

[0016] in, Represents the convolution kernel , a convolutional block consisting of a convolutional layer with a stride of 2 and a ReLU function, GAP represents the global average pooling layer, and FC represents the fully connected layer;

[0017] S2.2, Original image of the moment covered with ice At the same time, it is input into the shared feature encoder branch of the dynamic routing image enhancement autoencoder, which extracts common features such as mid- and high-level semantic structures and edge textures of the image, and gradually compresses the image size to reduce redundant information. The encoder consists of three layers of convolution blocks, each of which consists of It consists of convolution, batch normalization and ReLU functions, and some layers have maximum pooling operations, which ultimately output a multi-channel shared feature. ;

[0018] S2.3, the branch controller adopts the DecSwitch routing strategy, according to the output label of the noise type discriminator , activate one of the three decoder branches and convert the shared features output by the encoder into Routed to this decoder branch, when hour It is input into the denoising branch Decoder1, hour It is input into the fuzzy enhancement branch Decoder2, hour The data is input into the lightweight enhanced branch Decoder3, and the DecSwitch routing policy is defined as follows:

[0019] ;

[0020] S2.4, when the noise class discriminator outputs , shared features Enter the denoising branch Decoder1, which can enhance clear textures, edges and local structures. Its structure includes the UpConv upsampling convolution module and the ResBlock residual block. The residual block consists of convolution and ReLU activation functions. After passing through a three-channel output layer, the sigmoid activation function compresses the pixels to the [0,1] interval and finally outputs Image after time enhancement , the relevant calculation formula is as follows:

[0021]

[0022]

[0023] in, Represents the enhanced image Output from the denoising branch, represents the sigmoid activation function, express Convolution, UpConv upsampling convolution module includes ConvTranspose2D transposed convolution and convolution, represents input features, BN represents batch normalization;

[0024] S2.5, when the noise class discriminator outputs When shared features Entering the blur enhancement branch Decoder2, it passes through multiple UpConv blocks, convolution blocks, ReLU functions, and CEB (Contrast Enhance Block) contrast enhancement blocks, and finally outputs the enhanced image through the sigmoid function , the relevant calculation formula is as follows:

[0025]

[0026]

[0027] in, represents the input features, SE represents the channel attention mechanism for enhancing key channels;

[0028] S2.6, when the noise class discriminator outputs When , the shared features Enter the light enhancement branch Decoder3, which only makes slight adjustments to brightness and contrast. After multiple UpConv modules for upsampling, the The convolution and sigmoid functions output the enhanced image.

[0029] As a further preferred embodiment of the ice detection method based on multi-source data feature interaction of the present invention, in step S3, obtaining local ice features and global ice features specifically includes the following steps:

[0030] S3.1, the enhanced image Input dual-branch feature extraction network Terrafuse-BiNet, in the deformable multi-scale detail feature extraction branch, First, it is input into the basic feature extraction layer, which includes ordinary convolution, batch normalization operation, ReLU activation function and CBAM convolution attention module to obtain primary features. ,The specific feature extraction process can be expressed by the following formula:

[0031]

[0032] The primary features The input includes a gradient-guided and regularized deformable convolutional network, which allows for The input feature map is dynamically sampled, and its sampling position is determined by a fixed sampling offset. and learnable offsets Together, the calculation process of the output features is expressed as:

[0033]

[0034] in, Represents the position in the output feature map The characteristic response at represents the primary feature map of the input, Indicates the The convolution kernel weight of the sampling point, is the number of sampling points, is the weight coefficient guided by the input gradient, satisfying , the specific formula is as follows:

[0035]

[0036] in, Indicates the position on the input feature map The gradient amplitude at , To control the hyperparameters of local enhancement, which are used to control the degree of response to local gradient changes, an L2 regularization term is introduced to constrain the offset amplitude. The specific constraints are:

[0037]

[0038] in, represents the offset regularization loss, express The L2 norm square of the final output structure adaptive feature map ;

[0039] S3.2, The input dilated spatial pyramid pooling module extracts multi-scale context information. The module consists of three parallel dilated convolution branches with different dilation rates. The calculation process of the branches is expressed as:

[0040]

[0041] in, Indicates different void ratios, The void ratio is of Dilated convolution operation, Represents the local features extracted at a certain void rate, and the three sets of output features are spliced in the channel dimension to generate a detail feature map , the calculation process is expressed as:

[0042]

[0043] Among them, Concat represents the channel-wise splicing operation. They are the local feature representations of the three parallel dilated convolution branches, and finally the residual connection is performed to output the local features of the ice cover. The specific operations are as follows:

[0044]

[0045] S3.3, the enhanced image The DEM data is input into the height difference guided attention perception branch of the dual-branch feature extraction network Terrafuse-BiNet, and the channel dimension is first spliced to generate a terrain joint feature map. , which is expressed as:

[0046]

[0047] in, Represents the DEM elevation data at time t, and then introduces the normalized coordinate encoding to convert the normalized row and column coordinate tensor of each pixel point into Splice to In the spatial perception joint feature map :

[0048]

[0049]

[0050]

[0051] in, and They encode horizontal and vertical position information respectively. Normalize means linearly scaling the row and column indices to the interval [0,1]. Meshgrid generates the corresponding coordinate tensor. 、 Represents height and width respectively, 、 Respectively represent the horizontal and vertical coordinates; Input lightweight convolutional encoder, the encoder consists of two Convolution blocks, each of which contains convolution layers, batch normalization, and ReLU activation functions to extract preliminary spatial terrain fusion features , and then through Convolution will Mapping generates query matrix , key matrix , value matrix :

[0052]

[0053] Introducing height difference spatial deviation function As a position enhancement item, the attention score matrix is calculated, and the calculation process is expressed as:

[0054]

[0055] in, and They are The query and key vectors at represents the transpose operation, is the key vector dimension, and The pixel positions and The normalized coordinate difference of is the pixel position The elevation difference, express 、 and A linear weighted combination of :

[0056]

[0057] in, 、 、 is an adjustable hyperparameter with a range of [0.1, 10], and the attention score matrix The attention weight is obtained after normalization by the softmax function , and then the focus position Features According to weight Perform weighted summation to generate position Attention aggregation results :

[0058]

[0059] Use residual connection pairs and Processing is performed to obtain the global characteristics of ice cover :

[0060]

[0061] in, ∈(0,1) is the adjustable fusion coefficient, which is used to adjust and The proportion balance between.

[0062] As a further preferred solution of the ice detection method based on multi-source data feature interaction of the present invention, in step S4, obtaining meteorological time series features specifically includes the following steps:

[0063] S4.1. Input the multidimensional meteorological data at each moment into the meteorological feature extraction module TRAM-Net. First, it enters the variable weighting unit, which consists of a linear mapping layer, a nonlinear activation layer, and a task-guided attention unit. After linear transformation of the vector weight matrix and the bias term, an intermediate representation is generated through tanh activation. Then, the similarity is calculated with the learnable vector and normalized to obtain the variable importance weight at the current moment. , used to adjust dynamic variables, the calculation process is as follows:

[0064]

[0065]

[0066] in, is the trainable mapping matrix, is the bias term, is the task attention vector, represents the transpose operation, Indicates the The meteorological characteristic vector of time steps, for Time-weighted meteorological characteristics, The value range is (0,1);

[0067] S4.2. Using Position Vectors Weighted meteorological characteristics Inject position information to form an embedding vector with time and space awareness ,Will Input the local temporal dynamic part and use a one-dimensional convolution sliding window with a kernel of 3 to extract the local dynamic features within the window. The calculation process is expressed as:

[0068]

[0069]

[0070] in, Indicates the convolution kernels, express Local dynamic characteristics of the moment;

[0071] S4.3. Based on weighted meteorological characteristics Defined by The time is the center and the length is Sliding window , and calculate Moment The average value of the feature within the sliding window The L2 norm square between and the local meteorological variability rate , which is used to indicate the degree of deviation between the current meteorological state and other time points within the window. The variation rate is defined as follows:

[0072] ;

[0073] S4.4. Local dynamic features Input mutation enhanced attention module, additionally constructed 、 、 , and Introducing attention score as attention bias The module is shifted to focus on the key change moments, and finally the weight normalization and weighted aggregation are completed. The calculation process is as follows:

[0074]

[0075]

[0076]

[0077] in, , , represents the mapping matrix, represents the transpose operation, represents the empirical scaling factor in the self-attention mechanism, represents the mutation guidance coefficient, Indicates the The attention output after fusion of time steps, represents the total length of the meteorological time series;

[0078] S4.5. Local meteorological variability at all times Perform softmax normalization to obtain the time weight coefficient , and Perform weighted sum processing to obtain meteorological time series characteristics of uniform length , the specific calculation process is as follows:

[0079]

[0080]

[0081] As a further preferred embodiment of the ice detection method based on multi-source data feature interaction of the present invention, in step S5, feature fusion is performed on the local ice features, global ice features, and meteorological time series features extracted in S2 and S3 to obtain ice fusion features, which specifically includes the following steps:

[0082] S5.1. Global ice cover characteristics , local characteristics of ice cover and meteorological time series characteristics Input three independent fully connected layers respectively for unified encoding mapping. The linear mapping process of each feature path is:

[0083]

[0084] in, Represents a linear mapping operation, outputting the encoded ice coverage global features of uniform dimension , local characteristics of ice cover and meteorological time series characteristics ;

[0085] S5.2. Encoded features and The spatial feature interaction is performed through the bidirectional cross-modal attention mechanism, and its calculation process is expressed as:

[0086]

[0087]

[0088]

[0089] in, Represents a multi-head attention mechanism, outputting the spliced spatial fusion features , then and meteorological time series characteristics The attention mechanism is performed twice with each other as input to achieve two-way information interaction and fusion, and the initial fusion features of uniform length are obtained after splicing and linear mapping. ;

[0090] S5.3. Initial fusion features Input lightweight Transformer encoder, after normalization, multi-head attention, feedforward network and residual connection, the final output is fusion feature .

[0091] As a further preferred embodiment of the ice detection method based on multi-source data feature interaction of the present invention, in step S6, obtaining the ice type and ice thickness level specifically includes the following steps:

[0092] S6.1. Fusion features Input the multi-task prediction module and perform normalization. The features of each channel are first extracted by global average pooling to obtain the mean and standard deviation, and then normalized within the channel dimension. The channel offset term is introduced in the normalization process for scale adjustment, and then the unified scale feature tensor is output. ;

[0093] S6.2, Input residual fusion encoder, which consists of three residual blocks stacked in sequence. Each residual block contains two fully connected layers, a GELU activation function and an SE channel attention module. The features are sent to the SE channel attention module after GELU activation, and global average pooling, two-layer full connection and sigmoid normalization are performed in sequence to generate channel weights. The channel weights are multiplied by the original features, and then residual connection is performed to obtain the output of the first residual block. The output of the first residual block is used as the input of the second residual block. After the three layers of residual blocks act in sequence, the enhanced feature tensor after multi-layer fusion is output. ;

[0094] S6.3, Input multi-head self-attention layer, which consists of two groups of attention heads, used to extract ice type features and ice thickness level features respectively. Each group of attention structures is generated by linear transformation 、 、 , and perform similarity calculation on the spatial dimension to obtain the attention score, then perform weighted summation on the value vector, introduce the task bias matrix into each group of attention structure, and the bias matrix participates in the correction of the attention score to output the ice type features respectively Characteristics of ice thickness level ;

[0095] S6.4. Characteristics of ice cover types Characteristics of ice thickness level The lightweight prediction module is input separately, which consists of a global average pooling layer and two fully connected layers, where the ice type branch outputs the category prediction result. , ice thickness grade branch output grade distribution , the calculation process is as follows:

[0096]

[0097]

[0098] Among them, GAP represents the global average pooling operation, 、 is the fully connected weight matrix of the branch corresponding to the task, and the output result is finally determined by the maximum probability. In addition, the loss function adopts the task weighted form for joint training:

[0099]

[0100] in, 、 represents the cross entropy loss, 、 Indicates that the task weight can be adjusted to meet .

[0101] As a further preferred embodiment of the present invention, a method for detecting ice coating based on multi-source data feature interaction includes a line ice coating detection method, which includes: a preprocessing module, a dual-branch feature extraction module, a meteorological feature extraction module, a feature fusion module, and a multi-task prediction module;

[0102] Preprocessing module: The original transmission line ice-covered images are organized into ice-covered image sequences in chronological order. After normalization, they are input into the image enhancement autoencoder. The corresponding decoding branch is activated as needed to implement denoising, blur enhancement, or lightweight enhancement to obtain the enhanced image.

[0103] Dual-branch feature extraction module: The enhanced image is input into the deformable multi-scale detail feature extraction branch of the dual-branch feature extraction network Terrafuse_BiNet. Structural adaptive features are extracted and fused with dilated convolution to obtain multi-scale local information to form a local feature map of ice cover. Simultaneously, in the elevation difference-guided attention perception branch, the enhanced image is spliced with DEM data and combined with coordinate encoding to construct spatial joint features. After attention calculation and aggregation, a global feature of ice cover that integrates terrain information is generated.

[0104] Meteorological feature extraction module: The multidimensional meteorological data at each moment is input into the meteorological feature extraction module TRAM-Net, which extracts key variables and integrates time and location information. Local convolution is used to perceive short-term trends, and the local meteorological variability rate of the sliding window is combined to capture meteorological fluctuation characteristics. Finally, meteorological time series features are generated through global enhancement and weighted aggregation.

[0105] Feature fusion module: After encoding the global ice cover features, local ice cover features, and meteorological time series features, the modules undergo bidirectional cross-scale attention fusion, time series interaction fusion, and Transformer encoding, ultimately obtaining ice cover fusion features of a unified dimension.

[0106] Multi-task prediction module: The ice cover fusion features are normalized and input into the residual fusion encoder to extract enhanced features. The task-aware multi-head attention module then generates ice cover type features and ice cover thickness level features, which are respectively sent to the lightweight prediction module to finally output the ice cover type and ice cover thickness level prediction results.

[0107] Compared with the prior art, the present invention adopts the above technical solution and has the following beneficial effects:

[0108] 1. The line icing detection method of the present invention adopts a multi-branch image enhancement strategy of noise discrimination and routing decoding. It automatically identifies the type of image noise through a dynamic routing image enhancement autoencoder and activates the corresponding denoising, blur enhancement, or lightweight enhancement branches. It effectively enhances dark information and improves contrast. It also addresses various image quality degradation issues such as Gaussian noise, blur, defocus, and slight interference, enhances image structural details, improves image quality in complex environments, avoids redundant calculations, and maintains feature integrity. It is significantly superior to traditional single-branch enhancement methods and provides a clear, stable, and high-quality image foundation for subsequent icing feature extraction.

[0109] 2. The line ice detection method of the present invention constructs a dual-branch Terrafuse-BiNet network structure, achieving dual perception capabilities of complex terrain and local details. This structure exhibits stronger spatial adaptability in dealing with elevation fluctuations, perspective offsets, and other conditions, significantly improving the texture recognition ability in ice-covered areas. This effectively solves the problems of traditional methods' sensitivity to terrain disturbances and unstable detection accuracy, providing more robust technical support for comprehensive perception of power transmission lines in mountainous areas.

[0110] 3. The line icing detection method of the present invention effectively improves the model's ability to identify key meteorological variables and respond to sudden climate changes by introducing the TRAM-Net meteorological feature extraction module. This method can accurately capture the impact signals of meteorological factors in the temporal evolution process, enhance the detection system's modeling depth of complex climate dynamics, and significantly improve the prediction accuracy of the icing process and the accuracy of early warning. BRIEF DESCRIPTION OF THE DRAWINGS

[0111] Figure 1 This is a flow chart of the line icing detection method of the present invention;

[0112] Figure 2 This is a structural diagram of the overall model of the line icing detection method of the present invention;

[0113] Figure 3 Schematic diagram of the module structure of the image enhancement autoencoder in the method of the present invention;

[0114] Figure 4 This is a diagram showing the effect of an enhanced image output by the image enhancement autoencoder in the method of the present invention;

[0115] Figure 5 This is a diagram showing the impact of meteorological time series characteristics on model performance in the line icing detection method of the present invention;

[0116] Figure 6 This is a schematic diagram comparing the impact of meteorological time series characteristics on classification results in the present invention; Figure 6 (a) is the classification result diagram of the present invention without adding meteorological time series features; Figure 6(b) is the classification result diagram after adding meteorological time series features in the present invention;

[0117] Figure 7 This is a comparison curve of the accuracy of the Terrafuse-BiNet model comparison experiment of the present invention; DETAILED DESCRIPTION

[0118] In order to make the purpose, technical solutions and advantages of the present invention clearer, the technical solutions of the application are further elaborated in detail below with reference to the accompanying drawings. The described embodiments are only a part of the embodiments involved in the present invention. All non-innovative embodiments of other researchers in this field on this embodiment fall within the scope of protection of the present invention. At the same time, the step numbers in the embodiments of the present invention are only set for the convenience of explanation and description, and do not limit the order between the steps. The execution order of each step in the embodiment can be adaptively adjusted according to the understanding of those skilled in the art.

[0119] In one embodiment of the present invention, a line icing detection method is as follows: Figure 1 As shown, the following steps are included:

[0120] S1. Collect the actual original ice-covered images of the transmission lines through monitoring cameras on the transmission lines, organize the images into image sequences in chronological order, and simultaneously obtain DEM data and meteorological data;

[0121] S2, input the normalized ice-covered image into the dynamic routing image enhancement autoencoder and output the enhanced image;

[0122] S2.1、 Original ice cover image at the moment Input the noise type discriminator branch of the dynamic routing image enhancement autoencoder. After passing through the convolution layer, global average pooling layer and a fully connected layer within the branch, the softmax function outputs the category label with the largest probability value. , Indicates that the image contains obvious Gaussian noise. Indicates that the image is blurry or has low contrast. Indicates that the image does not have too much noise and only needs light enhancement. The calculation process is expressed as:

[0123]

[0124] in, Represents the convolution kernel , a convolutional block consisting of a convolutional layer with a stride of 2 and a ReLU function, GAP represents the global average pooling layer, and FC represents the fully connected layer;

[0125] S2.2, Original image of the moment covered with ice At the same time, it is input into the shared feature encoder branch of the dynamic routing image enhancement autoencoder, which extracts common features such as mid- and high-level semantic structures and edge textures of the image, and gradually compresses the image size to reduce redundant information. The encoder consists of three layers of convolution blocks, each of which consists of It consists of convolution, batch normalization and ReLU functions, and some layers have maximum pooling operations, which ultimately output a multi-channel shared feature. ;

[0126] S2.3, the branch controller adopts the DecSwitch routing strategy, according to the output label of the noise type discriminator , activate one of the three decoder branches and convert the shared features output by the encoder into Routed to this decoder branch, when hour It is input into the denoising branch Decoder1, hour It is input into the fuzzy enhancement branch Decoder2, hour The data is input into the lightweight enhanced branch Decoder3, and the DecSwitch routing policy is defined as follows:

[0127]

[0128] S2.4, when the noise class discriminator outputs , shared features Enter the denoising branch Decoder1, which can enhance clear textures, edges and local structures. Its structure includes the UpConv upsampling convolution module and the ResBlock residual block. The residual block consists of convolution and ReLU activation functions. After passing through a three-channel output layer, the sigmoid activation function compresses the pixels to the [0,1] interval and finally outputs Image after time enhancement , the relevant calculation formula is as follows:

[0129]

[0130]

[0131] in, Represents the enhanced image Output from the denoising branch, represents the sigmoid activation function, express Convolution, UpConv upsampling convolution module includes ConvTranspose2D transposed convolution and convolution, represents input features, BN represents batch normalization;

[0132] S2.5, when the noise class discriminator outputs When shared features Entering the blur enhancement branch Decoder2, it passes through multiple UpConv blocks, convolution blocks, ReLU functions, and CEB (Contrast Enhance Block) contrast enhancement blocks, and finally outputs the enhanced image through the sigmoid function , the relevant calculation formula is as follows:

[0133]

[0134]

[0135] in, represents the input features, SE represents the channel attention mechanism for enhancing key channels;

[0136] S2.6, when the noise class discriminator outputs When , the shared features Enter the light enhancement branch Decoder3, which only makes slight adjustments to brightness and contrast. After multiple UpConv modules for upsampling, the The convolution and sigmoid functions output the enhanced image.

[0137] S3. The enhanced image and DEM data are input into the dual-branch feature extraction network Terrafuse-BiNet in parallel. The deformable multi-scale detail feature extraction branch extracts edge textures, and the height difference guided attention perception branch fuses the terrain for spatial modeling. Finally, local and global ice cover features are obtained respectively. The specific steps include:

[0138] S3.1, the enhanced image Input dual-branch feature extraction network Terrafuse-BiNet, in the deformable multi-scale detail feature extraction branch, First, it is input into the basic feature extraction layer, which includes ordinary convolution, batch normalization operation, ReLU activation function and CBAM convolution attention module to obtain primary features. ,The specific feature extraction process can be expressed by the following formula:

[0139]

[0140] The primary features The input includes a gradient-guided and regularized deformable convolutional network, which allows for The input feature map is dynamically sampled, and its sampling position is determined by a fixed sampling offset. and learnable offsets Together, the calculation process of the output features is expressed as:

[0141]

[0142] in, Represents the position in the output feature map The characteristic response at represents the primary feature map of the input, Indicates the The convolution kernel weight of the sampling point, is the number of sampling points, is the weight coefficient guided by the input gradient, satisfying , the specific formula is as follows:

[0143]

[0144] in, Indicates the position on the input feature map The gradient amplitude at , To control the hyperparameters of local enhancement, which are used to control the degree of response to local gradient changes, an L2 regularization term is introduced to constrain the offset amplitude. The specific constraints are:

[0145]

[0146] in, represents the offset regularization loss, express The L2 norm square of the final output structure adaptive feature map ;

[0147] S3.2, The input dilated spatial pyramid pooling module extracts multi-scale context information. The module consists of three parallel dilated convolution branches with different dilation rates. The calculation process of the branches is expressed as:

[0148]

[0149] in, Indicates different void ratios, The void ratio is of Dilated convolution operation, Represents the local features extracted at a certain void rate, and the three sets of output features are spliced in the channel dimension to generate a detail feature map , the calculation process is expressed as:

[0150]

[0151] Among them, Concat represents the channel-wise splicing operation. They are the local feature representations of the three parallel dilated convolution branches, and finally the residual connection is performed to output the local features of the ice cover. The specific operations are as follows:

[0152]

[0153] S3.3, the enhanced image The DEM data is input into the height difference-guided attention perception branch of the Terrafuse-BiNet dual-branch feature extraction network. The channel dimension is first spliced to generate a terrain joint feature map. , which is expressed as:

[0154]

[0155] in, Represents the DEM elevation data at time t, and then introduces the normalized coordinate encoding to convert the normalized row and column coordinate tensor of each pixel point into Splice to In the spatial perception joint feature map :

[0156]

[0157]

[0158]

[0159] in, and They encode horizontal and vertical position information respectively. Normalize means linearly scaling the row and column indices to the interval [0,1]. Meshgrid generates the corresponding coordinate tensor. 、 Represents height and width respectively, 、 Respectively represent the horizontal and vertical coordinates; Input lightweight convolutional encoder, the encoder consists of two Convolution blocks, each of which contains convolution layers, batch normalization, and ReLU activation functions to extract preliminary spatial terrain fusion features , and then through Convolution will Mapping generates query matrix , key matrix , value matrix :

[0160]

[0161] Introducing height difference spatial deviation function As a position enhancement item, the attention score matrix is calculated, and the calculation process is expressed as:

[0162]

[0163] in, and respectively The query and key vectors at represents the transpose operation, is the key vector dimension, and The pixel positions and The normalized coordinate difference of is the pixel position The elevation difference, express 、 and A linear weighted combination of :

[0164]

[0165] in, 、 、 is an adjustable hyperparameter with a range of [0.1, 10], and the attention score matrix The attention weight is obtained after normalization by the softmax function , and then the focus position Features According to weight Perform weighted summation to generate position Attention aggregation results :

[0166]

[0167] Use residual connection pairs and Processing is performed to obtain the global characteristics of ice cover :

[0168]

[0169] in, ∈(0,1) is the adjustable fusion coefficient, which is used to adjust and The proportion balance between.

[0170] S4. Input meteorological data into the TRAM-Net meteorological feature extraction module, use the local meteorological variation rate to guide the attention bias, complete feature encoding and output meteorological time series features, specifically including the following steps:

[0171] S4.1. Input the multidimensional meteorological data at each moment into the TRAM-Net meteorological feature extraction module. First, it enters the variable weighting unit, which consists of a linear mapping layer, a nonlinear activation layer, and a task-guided attention unit. After linear transformation of the vector weight matrix and the bias term, an intermediate representation is generated through tanh activation. Subsequently, the similarity is calculated with the learnable vector and normalized to obtain the variable importance weight at the current moment. , used to adjust dynamic variables, the calculation process is as follows:

[0172]

[0173]

[0174] in, is the trainable mapping matrix, is the bias term, is the task attention vector, represents the transpose operation, Indicates the The meteorological characteristic vector of time steps, for Time-weighted meteorological characteristics, The value range is (0,1);

[0175] S4.2. Using Position Vectors Weighted meteorological characteristics Inject position information to form an embedding vector with time and space awareness ,Will Input the local temporal dynamic part and use a one-dimensional convolution sliding window with a kernel of 3 to extract the local dynamic features within the window. The calculation process is expressed as:

[0176]

[0177]

[0178] in, Indicates the convolution kernels, express Local dynamic characteristics of the moment;

[0179] S4.3. Based on weighted meteorological characteristics Defined by The time is the center and the length is Sliding window , and calculate Moment The average value of the feature within the sliding window The L2 norm square between and the local meteorological variability rate , which is used to indicate the degree of deviation between the current meteorological state and other time points within the window. The variation rate is defined as follows:

[0180] ;

[0181] S4.4. Local dynamic features Input mutation enhanced attention module, additionally constructed 、 、 , and Introducing attention score as attention bias The module is shifted to focus on the key change moments, and finally the weight normalization and weighted aggregation are completed. The calculation process is as follows:

[0182]

[0183]

[0184]

[0185] in, , , represents the mapping matrix, represents the transpose operation, represents the empirical scaling factor in the self-attention mechanism, represents the mutation guidance coefficient, Indicates the The attention output after fusion of time steps, represents the total length of the meteorological time series;

[0186] S4.5. Local meteorological variability at all times Perform softmax normalization to obtain the time weight coefficient , and Perform weighted sum processing to obtain meteorological time series characteristics of uniform length , the specific calculation process is as follows:

[0187]

[0188]

[0189] S5: After unified scale mapping and feature precoding, the local and global ice cover features and meteorological time series features are interactively fused through a bidirectional cross-modal attention mechanism and feature splicing operations, and the scale-aligned multimodal ice cover fusion features are finally output. The specific steps include:

[0190] S5.1. Global ice cover characteristics , local characteristics of ice cover and meteorological time series characteristics Input three independent fully connected layers respectively for unified encoding mapping. The linear mapping process of each feature path is:

[0191]

[0192] in, Represents a linear mapping operation, outputting the encoded ice coverage global features of uniform dimension , local characteristics of ice cover and meteorological time series characteristics ;

[0193] S5.2. Encoded features and The spatial feature interaction is performed through the bidirectional cross-modal attention mechanism, and its calculation process is expressed as:

[0194]

[0195]

[0196]

[0197] in, Represents a multi-head attention mechanism, outputting the spliced spatial fusion features , then and meteorological time series characteristics The attention mechanism is performed twice with each other as input to achieve two-way information interaction and fusion, and the initial fusion features of uniform length are obtained after splicing and linear mapping. ;

[0198] S5.3. Initial fusion features Input lightweight Transformer encoder, after normalization, multi-head attention, feedforward network and residual connection, the final output is fusion feature .

[0199] S6. Input the ice cover fusion features into the multi-task prediction module, where the residual fusion encoder extracts deep features. Then, the task-aware multi-head self-attention module extracts type features and thickness level features. Finally, the lightweight classifier is input to obtain the corresponding ice cover type and ice cover thickness level. Specifically, the following steps are included:

[0200] S6.1. Fusion features Input the multi-task prediction module and perform normalization. The features of each channel are first extracted by global average pooling to obtain the mean and standard deviation, and then normalized within the channel dimension. The channel offset term is introduced in the normalization process for scale adjustment, and then the unified scale feature tensor is output. ;

[0201] S6.2, Input residual fusion encoder, which consists of three residual blocks stacked in sequence. Each residual block contains two fully connected layers, a GELU activation function and an SE channel attention module. The features are sent to the SE channel attention module after GELU activation, and global average pooling, two-layer full connection and sigmoid normalization are performed in sequence to generate channel weights. The channel weights are multiplied by the original features, and then residual connection is performed to obtain the output of the first residual block. The output of the first residual block is used as the input of the second residual block. After the three layers of residual blocks act in sequence, the enhanced feature tensor after multi-layer fusion is output. ;

[0202] S6.3, Input multi-head self-attention layer, which consists of two groups of attention heads, used to extract ice type features and ice thickness level features respectively. Each group of attention structures is generated by linear transformation 、 、 , and perform similarity calculation on the spatial dimension to obtain the attention score, then perform weighted summation on the value vector, introduce the task bias matrix into each group of attention structure, and the bias matrix participates in the correction of the attention score to output the ice type features respectively Characteristics of ice thickness level ;

[0203] S6.4. Characteristics of ice cover types Characteristics of ice thickness level The lightweight prediction module is input separately, which consists of a global average pooling layer and two fully connected layers, where the ice type branch outputs the category prediction result. , ice thickness grade branch output grade distribution , the calculation process is as follows:

[0204]

[0205]

[0206] Among them, GAP represents the global average pooling operation, 、 is the fully connected weight matrix of the branch corresponding to the task, and the output result is finally determined by the maximum probability. In addition, the loss function adopts the task weighted form for joint training:

[0207]

[0208] in, 、 represents the cross entropy loss, 、 Indicates that the task weight can be adjusted to meet .

[0209] The overall structure of the present invention is as follows Figure 2 As shown in the figure, it contains five core modules: preprocessing module, dual-branch feature extraction module, meteorological feature extraction module, feature fusion module and multi-task prediction module. Its collaborative process is as follows:

[0210] First, the original ice cover image is obtained by the monitoring equipment and input into the preprocessing module to organize it into an ice cover image sequence in chronological order and normalize it. Then it is input into the image enhancement autoencoder, whose structure is as follows: Figure 3 As shown, according to the output result of the noise type discriminator, the corresponding decoding branch is activated as needed to achieve denoising, blur enhancement or light enhancement to obtain the enhanced image. Figure 4 This is a comparison diagram of the enhanced image and the original ice-covered image.

[0211] The enhanced image is input into the deformable multi-scale detail feature extraction branch of the Terrafuse_BiNet dual-branch feature extraction network. Structural adaptive features are extracted and fused with dilated convolution to obtain multi-scale local information to form a local feature map of ice cover. At the same time, in the elevation difference guided attention perception branch, the enhanced image is spliced with DEM data and combined with coordinate encoding to construct spatial joint features. After attention calculation and aggregation, a global feature of ice cover that integrates terrain information is generated.

[0212] At the same time, multi-dimensional meteorological data at each moment is input into the TRAM-Net meteorological feature extraction module, which extracts key variables and integrates time and location information. Local convolution is used to perceive short-term trends, and the sliding window mutation rate is combined to capture meteorological fluctuation characteristics. Finally, meteorological time series features are generated through global enhancement and weighted aggregation.

[0213] Then, the global ice cover features, local ice cover features and meteorological time series features are encoded separately, and then bidirectional cross-scale attention fusion, time series interaction fusion and Transformer encoding are carried out in sequence to finally obtain the ice cover fusion features of unified dimension.

[0214] Finally, the ice fusion features are normalized and input into the residual fusion encoder to extract enhanced features. Then, the task-aware multi-head attention module is used to generate branch features of ice type and ice thickness level, which are respectively sent to the lightweight prediction module, and finally the ice type and ice thickness level prediction results are output.

[0215] In addition, the TRAM-Net meteorological feature extraction module is subjected to ablation experiments. Experiments are conducted in models with and without meteorological time series features, and the results are recorded. Figure 5 、 Figure 6 (a) and Figure 6 As shown in (b), the detailed results are shown in Table 1 below:

[0216] Table 1

[0217] Whether to add meteorological time series features Acer F1 Weighted F1 score Balanced Accuracy Matthews correlation coefficient Accuracy no 88.1% 89.5% 88.3% 85% 89% yes 89.5% 90.6% 89.7% 85.5% 90.6%

[0218] It can be seen that the accuracy of the model is significantly improved after adding meteorological time series features, especially the prediction of rime and hoarfrost is more accurate.

[0219] At the same time, advanced models such as MedViT, ResNet-101, EDPNet and YOLO12 were used to train and test on the ice type recognition dataset, and the performance of these models and the accuracy change curves on the validation set were recorded. They were compared with the Terrafuse-BiNet method of the present invention. The experimental results are shown in the figure below. Figure 7 TerraFuse-BiNet achieves optimal Top-1 accuracy, precision, recall, and F1 composite scores while maintaining moderate floating-point operations per second and good inference speed, demonstrating its structural design advantages and deployment balance in complex ice-covered image recognition tasks. Detailed results are shown in Table 2 below:

[0220] Table 2

[0221] Top-1 Accuracy (%) Accuracy (%) Recall rate (%) F1 score (%) Floating point operations per second (G) Reasoning time (s) MedViT 88.20 87.41 86.47 86.99 1.1 0.0015 ResNet-101 87.60 87.00 85.92 86.44 0.65 0.0012 EDPNet 85.90 85.48 83.66 84.45 0.35 0.0007 YOLO12 83.89 84.52 82.90 83.71 0.2 0.0005 Terrafuse-BiNet (method of the present invention) 89.01 88.31 86.97 87.1 0.8 0.001

[0222] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. An ice detection method based on multi-source data feature interaction, characterized in that: The following steps are involved: S1. Collect the actual original ice-covered images of the transmission lines through monitoring cameras on the transmission lines, organize the images into image sequences in chronological order, and simultaneously obtain DEM data and meteorological data; S2, input the normalized ice-covered image into the dynamic routing image enhancement autoencoder and output the enhanced image; S3. The enhanced image and DEM data are input into the dual-branch feature extraction network Terrafuse-BiNet in parallel. The deformable multi-scale detail feature extraction branch extracts edge textures, and the height difference guided attention perception branch fuses the terrain for spatial modeling, respectively obtaining local ice cover features and global ice cover features. In step S3, extracting local features and global features of ice coverage includes the following steps: S3.1, the enhanced image Input dual-branch feature extraction network Terrafuse-BiNet, in the deformable multi-scale detail feature extraction branch, First, it is input into the basic feature extraction layer, which includes ordinary convolution, batch normalization operation, ReLU activation function and CBAM convolution attention module to obtain primary features. The specific feature extraction process can be expressed by the following formula: The primary features The input includes a gradient-guided and regularized deformable convolutional network. The deformable convolution allows dynamic sampling of the input feature map at each output position p0, where the sampling position is determined by a fixed sampling offset p. k and the learnable offset Δp k Together, the calculation process of the output features is expressed as: in, represents the feature response at position p0 in the output feature map, Represents the primary feature map of the input, w(k) represents the convolution kernel weight of the kth sampling point, N is the number of sampling points, α(p0) is the weight coefficient guided by the input gradient, satisfying α(p0)∈[1,3]. The specific formula is as follows: in, represents the gradient amplitude at position p0 on the input feature map, γ is a hyperparameter that controls local enhancement, which is used to control the degree of response to local gradient changes, and introduces the L2 regularization term to constrain the offset amplitude. The specific constraints are: Among them, L offset represents the offset regularization loss, Indicates p k The L2 norm square of the final output structure adaptive feature map S3.2, The input dilated spatial pyramid pooling module extracts multi-scale context information. The dilated spatial pyramid pooling module consists of three parallel dilated convolution branches with different dilation rates. The calculation process of the branches is expressed as follows: Among them, r represents different void ratios, represents a 3×3 dilated convolution operation with a dilation rate of r. Represents the local features extracted at a certain void rate, and the three sets of output features are spliced in the channel dimension to generate a detail feature map The calculation process is expressed as: Among them, Concat represents the channel-wise splicing operation. They are the local feature representations of the three parallel dilated convolution branches, and finally the residual connection is performed to output the local features of the ice cover. The specific operations are as follows: S3.3, the enhanced image The DEM data is input into the height difference guided attention perception branch of the Terrafuse-BiNet dual-branch feature extraction network, and the channel dimension is first spliced to generate the terrain joint feature map X t , which is expressed as: Among them, D t Represents the DEM elevation data at time t, and then introduces the normalized coordinate encoding to convert the normalized row and column coordinate tensor X of each pixel point coord ,Y coord Splice to X t In the spatial perception joint feature map Among them, X coord and Y coord They encode horizontal and vertical position information respectively. Normalize means linearly scaling the row and column indexes to the interval [0,1]. Meshgrid generates the corresponding coordinate tensor. H and W represent the height and width respectively, and x and y represent the horizontal and vertical coordinates respectively. coord Input lightweight convolutional encoder, the encoder consists of two 3×3 convolutional blocks, each of which contains a convolutional layer, batch normalization and ReLU activation function to extract preliminary spatial terrain fusion features Then, the 1×1 convolution is performed The mapping generates the query matrix Q, key matrix K, and value matrix V: The height difference spatial deviation function ψ(·) is introduced as a position enhancement term to calculate the attention score matrix. The calculation process is expressed as: Among them, Q i and K j are the query and key vectors at i and j, respectively, representing the transposition operation, d k is the bond vector dimension, Δx ij and Δy ij are the normalized coordinate differences of pixel positions i and j, Δh ij is the elevation difference between pixel positions i and j, and ψ(·) represents Δx ij , Δy ij and Δh ij A linear weighted combination of : ψ(Δx ij ,Dy ij ,Dh ij )=α1·Δx ij 2 +α2·Δy ij 2 +α3·Δh ij 2 Among them, α1, α2, and α3 are adjustable hyperparameters with a range of [0.1, 10]. The attention score matrix S ij After normalization by the softmax function, the attention weight a is obtained ij , then the feature V of the focus position j j According to the weight a ij Perform weighted summation to generate the attention aggregation result Z at position i i : Use residual connection to Z i and Processing is performed to obtain the global characteristics of ice cover Among them, λ∈(0,1) is the adjustable fusion coefficient, which is used to adjust Z i and The balance between the proportions; S4. Input meteorological data into the meteorological feature extraction module TRAM-Net, use the local meteorological variation rate to guide the attention bias, complete feature encoding and output meteorological time series features; S5: After unified scale mapping and feature precoding, the local and global features of ice cover and meteorological time series features are interactively fused through a bidirectional cross-modal attention mechanism and feature splicing operations to output scale-aligned multimodal ice cover fusion features. S6. The ice cover fusion features are input into the multi-task prediction module, where the residual fusion encoder extracts deep features, and then the task-aware multi-head self-attention module extracts type features and thickness level features, which are input into the lightweight classifier to obtain the corresponding ice cover type and ice cover thickness level.

2. The ice detection method based on multi-source data feature interaction according to claim 1 is characterized by: In step S2, obtaining an enhanced image includes the following steps: S2.1, the original ice cover image I at time t t The input is the noise type discriminator branch of the dynamic routing image enhancement autoencoder. After passing through the convolutional layer, global average pooling layer, and a fully connected layer within the branch, the softmax function outputs the category label T with the maximum probability value. T = 0 indicates that the image contains obvious Gaussian noise, T = 1 indicates that the image is blurred or has low contrast problems, and T = 2 indicates that the image has no excessive noise and only requires light enhancement. The calculation process is expressed as follows: T=argmax(softmax(FC(GAP(ConvBlock(I t ))))),T∈{0,1,2} Among them, ConvBlock represents a convolution block consisting of a convolution layer with a convolution kernel of 3×3 and a stride of 2 and a ReLU function, GAP represents a global average pooling layer, and FC represents a fully connected layer; S2.2, original ice image I at time t t At the same time, it is input into the shared feature encoder branch of the dynamic routing image enhancement autoencoder, which extracts common features of the image, including mid- and high-level semantic structures and edge textures, and gradually compresses the image size to reduce redundant information. The encoder consists of three layers of convolution blocks. Each convolution block consists of 3×3 convolution, batch normalization, and ReLU function. Some layers have maximum pooling operations, and finally output a multi-channel shared feature F; S2.

3. The branch controller uses the DecSwitch routing strategy. According to the output label T of the noise type discriminator, it activates one of the three decoder branches and routes the shared feature F output by the encoder to the decoder branch. When T = 0, F is input to the denoising branch Decoder1; when T = 1, F is input to the fuzzy enhancement branch Decoder2; when T = 2, F is input to the lightweight enhancement branch Decoder3. The DecSwitch routing strategy is defined as follows: S2.4, when the noise class discriminator outputs T = 0, the shared feature F enters the denoising branch Decoder1, which can enhance clear textures, edges and local structures. Its structure includes the upsampling convolution module UpConv and the residual block ResBlock. The residual block consists of convolution and ReLU activation functions. After passing through a three-channel output layer, the sigmoid activation function compresses the pixels to the [0, 1] interval, and finally outputs the enhanced image at time t. The relevant calculation formula is as follows: ResBlock(x)=x+BN(Conv 3×3 (ReLU(BN(Conv 3×3 (x))))) in, Represents the enhanced image Output from the denoising branch, σ(·) represents the sigmoid activation function, Conv 3×3 Represents 3×3 convolution. The UpConv upsampling convolution module includes ConvTranspose2D transposed convolution and 3×3 convolution. x represents input features, and BN represents batch normalization. S2.5, when the noise class discriminator outputs T = 1, the shared feature F enters the fuzzy enhancement branch Decoder2, passes through multiple UpConv blocks, convolution blocks, ReLU functions and CEB (Contrast Enhance Block) contrast enhancement blocks, and finally outputs the enhanced image through the sigmoid function. The relevant calculation formula is as follows: CEB(y)=y+SE(Conv 3×3 (ReLU(Conv 3×3 (and)))) Where y represents the input feature, SE represents the channel attention mechanism for enhancing the key channel; S2.

6. When the noise class discriminator outputs T = 2, the shared feature F is input into the lightweight enhancement branch Decoder3, and only slight brightness and contrast adjustments are made. F is upsampled through multiple UpConv modules, and finally the enhanced image is output through 3×3 convolution and sigmoid function.

3. The ice detection method based on multi-source data feature interaction according to claim 1 is characterized in that: In step S4, extracting meteorological time series features includes the following steps: S4.

1. Input the multidimensional meteorological data at each moment into the TRAM-Net meteorological feature extraction module. First, it enters the variable weighting unit, which consists of a linear mapping layer, a nonlinear activation layer, and a task-guided attention unit. After linear transformation of the vector weight matrix and the bias term, an intermediate representation is generated through tanh activation. Subsequently, the similarity is calculated with the learnable vector and normalized to obtain the variable importance weight ξ at the current moment. t , used to adjust dynamic variables, the calculation process is as follows: ξ t =σ(q v ·tanh(E t ·W v +b v )) Among them, W v is the trainable mapping matrix, b v is the bias term, q v is the task attention vector, representing the transposition operation, E t represents the meteorological feature vector at the t-th time step, is the weighted meteorological characteristic at time t, ξ t The value range is (0,1); S4.

2. Use position vector PE(t) to weight the meteorological characteristics Inject position information to form an embedding vector with time and space awareness Will Input the local temporal dynamic part and use a one-dimensional convolution sliding window with a kernel of 3 to extract the local dynamic features within the window. The calculation process is expressed as: Among them, W c s represents the sth convolution kernel, Z t Represents the local dynamic characteristics at time t; S4.

3. Based on weighted meteorological characteristics Define a sliding window [tn,t+n] with a length of 2n+1 and centered at time t, and calculate the time m. The average value of the feature within the sliding window The L2 norm square between and the local meteorological variability δ t , which is used to indicate the degree of deviation between the current meteorological state and other time points within the window. The variation rate is defined as follows: S4.4, local dynamic feature Z t Input mutation enhanced attention module, construct Q, K, V, and transform δ t Introducing the attention score A as an attention bias h,f The module is shifted to focus on the key change moments, and finally the weight normalization and weighted aggregation are completed. The calculation process is as follows: w h,f =softmax(A h,f ) Among them, W q , W k , W v represents the mapping matrix, represents the transpose operation, represents the empirical scaling factor in the self-attention mechanism, represents the mutation guidance coefficient, G h represents the attention output after fusion at the hth time step, and L represents the total length of the meteorological time series; S4.

5. Local meteorological variability δ for all time periods t Perform softmax normalization to obtain the time weight coefficient ω t , and for G h Perform weighted sum processing to obtain the meteorological time series characteristics F of uniform length weather , the specific calculation process is as follows: oh t =softmax(δ t ) 4. The ice detection method based on multi-source data feature interaction according to claim 1 is characterized in that: In step S5, the ice fusion feature is obtained, which includes the following steps: S5.

1. Set the ice cover global feature F global 、 Local characteristics of ice cover F local and meteorological time series characteristics F weather Input three independent fully connected layers respectively for unified encoding mapping. The linear mapping process of each feature path is: Among them, FC i (·) represents a linear mapping operation, which outputs the encoded ice coverage global features of uniform dimension. Local characteristics of ice cover and meteorological time series characteristics S5.

2. Encoded features and The spatial feature interaction is performed through the bidirectional cross-modal attention mechanism, and its calculation process is expressed as: F spatial =Concat(A g-t ,A t-g ) Among them, MHA(·) represents the multi-head attention mechanism, which outputs the spliced spatial fusion feature F spatial , then F spatial and meteorological time series characteristics The attention mechanism is performed twice with each other as input to achieve two-way information interaction and fusion, and the preliminary fusion feature F of uniform length is obtained after splicing and linear mapping. fused ; S5.3, the initial fusion feature F fused Input the lightweight Transformer encoder, after normalization, multi-head attention, feedforward network and residual connection, the final output is the fusion feature F final .

5. The ice detection method based on multi-source data feature interaction according to claim 1 is characterized in that: In step S6, obtaining the corresponding ice type and ice thickness level includes the following steps: S6.

1. Fusion feature F final Input the multi-task prediction module and perform normalization. The features of each channel are first extracted by global average pooling to obtain the mean and standard deviation, and then normalized within the channel dimension. The channel offset term is introduced in the normalization process for scale adjustment, and then the unified scale feature tensor is output. S6.2, Input residual fusion encoder, which consists of three residual blocks stacked in sequence. Each residual block contains two fully connected layers, a GELU activation function and an SE channel attention module. After the features are activated by GELU, they are sent to the SE channel attention module, which performs global average pooling, two-layer full connection and sigmoid normalization in sequence to generate channel weights. The channel weights are multiplied by the original features, and then residual connection is performed to obtain the output of the first residual block. The output of the first residual block is used as the input of the second residual block. After the three layers of residual blocks act in sequence, the enhanced feature tensor F after multi-layer fusion is output. res ; S6.3, F res Input the multi-head self-attention layer, which consists of two groups of attention heads, used to extract ice type features and ice thickness level features respectively. Each group of attention structures generates Q, K, and V respectively through linear transformation, and performs similarity calculation on the spatial dimension to obtain attention scores. Then, the value vectors are weighted summed. The task bias matrix is introduced in each group of attention structures. After the bias matrix participates in the attention score correction, the ice type features are output respectively. Characteristics of ice thickness levels S6.

4. Characteristics of ice cover types Characteristics of ice thickness level They are input into the lightweight prediction module, which consists of a global average pooling layer and two fully connected layers, where the ice type branch outputs the category prediction result P type , ice thickness grade branch output grade distribution P level , the calculation process is as follows: Among them, GAP represents the global average pooling operation, W1 and W2 are the fully connected weight matrices of the branches corresponding to the task, and the output result is finally determined by the maximum probability. In addition, the loss function adopts the task weighted form for joint training: L total =λ1·L type +λ2·L thickness Among them, L type , L thickness represents the cross entropy loss, λ1 and λ2 represent the adjustable task weights, satisfying λ1+λ2=1.

6. The ice detection method based on multi-source data feature interaction according to claim 1 is characterized in that: It includes preprocessing module, dual-branch feature extraction module, meteorological feature extraction module, feature fusion module and multi-task prediction module: Preprocessing module: The original transmission line ice-covered images are organized into ice-covered image sequences in chronological order. After normalization, they are input into the image enhancement autoencoder. The corresponding decoding branch is activated as needed to implement denoising, blur enhancement, or lightweight enhancement to obtain the enhanced image. Dual-branch feature extraction module: The enhanced image is input into the deformable multi-scale detail feature extraction branch of the dual-branch feature extraction network Terrafuse_BiNet. Structural adaptive features are extracted and fused with dilated convolution to obtain multi-scale local information to form a local feature map of ice cover. Simultaneously, in the elevation difference-guided attention perception branch, the enhanced image is spliced with DEM data and combined with coordinate encoding to construct spatial joint features. After attention calculation and aggregation, a global feature of ice cover that integrates terrain information is generated. Meteorological feature extraction module: The multidimensional meteorological data at each moment is input into the meteorological feature extraction module TRAM-Net, which extracts key variables and integrates time and location information. Local convolution is used to perceive short-term trends, and the local meteorological variability rate of the sliding window is combined to capture meteorological fluctuation characteristics. Finally, meteorological time series features are generated through global enhancement and weighted aggregation. Feature fusion module: After encoding the global ice cover features, local ice cover features, and meteorological time series features, the modules undergo bidirectional cross-scale attention fusion, time series interaction fusion, and Transformer encoding, ultimately obtaining ice cover fusion features of a unified dimension. Multi-task prediction module: The ice cover fusion features are normalized and input into the residual fusion encoder to extract enhanced features. The task-aware multi-head attention module then generates ice cover type features and ice cover thickness level features, which are respectively sent to the lightweight prediction module to finally output the ice cover type and ice cover thickness level prediction results.

Images