Power transmission line micro-meteorological identification method and system based on multi-modal data fusion

Abstract

The application provides a power transmission line micro-meteorological identification method and system based on multi-modal data fusion, which comprises the following steps: extracting picture information sources by using a Swin Transformer network structure for real-time meteorological pictures in a power transmission line scene; extracting quantitative data information sources by using a standard Transformer structure according to micro-meteorological observation data collected in the real-time power transmission line scene; using a regional perception semantic fusion mechanism to align and aggregate deep semantics between the picture information sources and the quantitative data information sources, and outputting final features; and identifying weather condition states by using a classification head according to the final features, and outputting a probability distribution of real-time weather condition categories. The application takes into account the spatial modeling advantages of the picture information sources and the stable perception ability of the quantitative data information sources, uses a regional perception semantic fusion mechanism to improve the semantic alignment accuracy between the information sources, and is especially suitable for high-precision identification of weather conditions in complex environments.

Description

Technical Field

[0001] This invention belongs to the field of computer vision analysis technology, specifically relating to a method and system for identifying micro-meteorological conditions in power transmission lines based on multimodal data fusion. Background Technology

[0002] With the large-scale commissioning of ultra-high voltage long-distance power transmission projects, a large number of overhead power lines built in the field are subjected to extreme weather processes such as rain, snow and ice for a long time, and safety accidents such as conductor icing and galloping often occur, posing a serious threat to the safety of the equipment itself and the power grid.

[0003] Image processing technology has driven the advancement of visualization data mining and applications, enabling batch identification of information contained in images. This provides technical support for monitoring local weather conditions along power lines using visualization devices. Image classification is a commonly used method, training weather condition recognition models using power line image data. However, traditional image classification algorithms have weak expressive power and low recognition accuracy, making them difficult to adapt to power line scenarios with complex backgrounds. Deep learning algorithms, with their superior feature extraction capabilities and detection accuracy, are better suited for power line weather condition recognition.

[0004] In-depth research was conducted on image classification and multi-source data aggregation technologies. Image classification technology was used to aggregate multi-source data to identify micro-meteorological conditions of power lines, enabling intelligent monitoring of power line weather conditions, better promoting the intelligent construction of the power grid, and ensuring the safety of the power system.

[0005] Currently, mainstream research methods for identifying weather conditions on power lines can be divided into two main categories: one is image-based visual recognition methods, which mainly rely on image feature extraction networks to model meteorological features in the scene; the other is quantitative data modeling methods based on sensor data, which infer weather conditions by analyzing sensor data features such as temperature, humidity, wind speed, and illumination. The former has advantages such as strong intuitive interpretability and rich spatial information, but its recognition accuracy is limited by problems such as fuzzy weather condition representation and drastic feature changes; the latter has stability and sensitivity in quantitative analysis, but lacks the ability to perceive context at the visual level, and the classification boundaries are unclear.

[0006] To enhance models' ability to perceive complex weather conditions, multi-source aggregation methods have emerged in recent years, attempting to integrate image and quantitative data to characterize weather conditions from multiple perspectives. Among these, deep learning-based multi-source aggregation methods possess powerful nonlinear modeling capabilities, enabling them to learn implicit complementary relationships between different information sources, thereby improving classification accuracy. However, if image and quantitative data sources cannot be effectively aggregated, it becomes difficult to establish clear semantic relationships, leading to information redundancy and semantic shifts in the aggregation process, ultimately affecting the stability and accuracy of overall classification performance. Summary of the Invention

[0007] In view of the above-mentioned defects or improvement needs of existing technologies, this invention proposes a method and system for identifying micro-meteorological conditions of transmission lines based on multimodal data fusion, which can improve the accuracy of semantic alignment between information sources.

[0008] To achieve the above objectives, according to one aspect of the present invention, a method for identifying micro-meteorological conditions in transmission lines based on multimodal data fusion is provided, comprising: The information source of real-time meteorological images of power transmission lines is extracted using the Swin Transformer network structure; the Swin Transformer network structure is trained by constructing a meteorological visualization dataset of power transmission lines. Based on real-time micrometeorological observation data collected in the transmission line scenario, a standard Transformer structure is used to extract quantitative data information sources; the standard Transformer structure is trained by constructing a micrometeorological observation dataset for transmission lines. A region-aware semantic fusion mechanism is used to perform deep semantic alignment and aggregation between image information sources and quantitative data information sources, and output the final features. Based on the final features, a classification head is used to identify the weather condition status and output the probability distribution of the weather condition category in real time.

[0009] According to the above scheme, the region-aware semantic fusion mechanism specifically includes: A recursive gated convolutional structure is used to transform the image information source to obtain an enhanced image information source. Attention mechanisms are used to extract key channel information from quantitative data sources; A channel-guided aggregation mechanism is adopted to aggregate enhanced image information sources and key channel information, and output the final features. The final features include the spatial structure information of the image information sources and the global semantic information of the quantitative data information sources.

[0010] According to the above scheme, the classification header specifically includes: The final features are compressed into a fixed-length one-dimensional vector through global average pooling. The one-dimensional vector is transformed through a fully connected layer, and the probability distribution of the weather condition category is output through the Softmax function.

[0011] According to the above scheme, the Swin Transformer network structure extracts image information sources through the following steps: Meteorological images are divided into non-overlapping patches of fixed size and converted into vector representations through linear transformation to construct an initial image embedding sequence; A layered architecture is used to extract image information sources step by step.

[0012] According to the above scheme, in the layered architecture, each layer consists of multiple Swing Transformer modules, which internally alternately stack local window attention mechanisms and shifting window mechanisms; among them, The local window attention mechanism is used to divide the entire image into several regions, thereby reducing the complexity of attention calculation; the shift window mechanism is used to translate the shift window region between adjacent modules to establish cross-region information pathways.

[0013] According to the above scheme, each layer also includes a Patch Merging unit, which is used to gradually compress the resolution of the feature map through spatial downsampling and simultaneously expand the channel dimension to realize the extraction and expression of multi-scale semantic information.

[0014] According to the above scheme, the standard Transformer structure extracts quantitative data information sources through the following steps: The acquired micrometeorological observation data are uniformly mapped to a high-dimensional representation space to construct an initial quantitative data embedding sequence, and location information is injected into the quantitative data embedding sequence in the form of location encoding. Multiple stacked Transformer base modules are used to extract quantitative data information sources, which are used to reflect the deep expression of the dynamic changes in quantitative data.

[0015] According to the above scheme, each Transformer basic module includes a multi-channel attention sublayer and a feedforward transform sublayer; wherein... The multi-channel attention sublayer is used to compute multiple attention paths in parallel, capturing key correlations between variables in a quantitative data embedding sequence from different subspaces; The feedforward transformation sublayer is used to enhance the expressiveness of the representation through nonlinear mapping.

[0016] According to the above scheme, each Transformer basic module is equipped with residual connections and layer normalization mechanisms to alleviate the gradient vanishing problem in deep networks and improve the stability of training standard Transformer structures.

[0017] According to another aspect of the present invention, a micro-meteorological identification system for transmission lines based on multimodal data fusion is provided, comprising: The image information source extraction module is used to extract image information sources from real-time meteorological images of power transmission line scenarios using the SwinTransformer network structure; the SwinTransformer network structure is trained by constructing a meteorological visualization dataset of power transmission lines; The quantitative data information source extraction module is used to extract quantitative data information sources based on real-time micro-meteorological observation data collected in the transmission line scenario, using a standard Transformer structure; the standard Transformer structure is trained by constructing a micro-meteorological observation dataset for transmission lines. The aggregation module is used to perform deep semantic alignment and aggregation between image information sources and quantitative data information sources using a region-aware semantic fusion mechanism, and output the final features. The micro-weather recognition module is used to identify weather conditions based on the final features using a classification head, and outputs the probability distribution of real-time weather condition categories.

[0018] In summary, compared with the prior art, the above-described technical solutions conceived by this invention can achieve the following beneficial effects: It combines the spatial modeling advantages of image information sources with the stable perception capabilities of quantitative data information sources, and utilizes a regional perception semantic fusion mechanism to improve the accuracy of semantic alignment between information sources. It is especially suitable for high-precision recognition scenarios of weather conditions in complex environments. Attached Figure Description

[0019] Figure 1 This is a flowchart of a method provided in an embodiment of the present invention.

[0020] Figure 2 This is a schematic diagram of the region-aware semantic fusion mechanism provided in an embodiment of the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0022] In the description of this invention, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified. According to one aspect of the present invention, a method for identifying micro-meteorological conditions in transmission lines based on multimodal data fusion is provided, such as... Figure 1 As shown, it includes: S1. Extract image information sources from real-time meteorological images of power transmission lines using the Swin Transformer network structure; the Swin Transformer network structure is trained by constructing a meteorological visualization dataset of power transmission lines.

[0023] Among them, the meteorological visualization dataset for power transmission lines is obtained by acquiring meteorological images of power transmission line scenarios under historical conditions, removing invalid samples such as duplicates or blurry images, and classifying them according to meteorological states such as sunny, rainy, and snowy.

[0024] The Swin Transformer network architecture takes meteorological images of power transmission lines as input. First, it divides the meteorological images into fixed-size non-overlapping patches and transforms them into vectorized representations through linear transformation, constructing an initial image embedding sequence. Then, the model employs a hierarchical architecture to progressively extract image features. Each layer consists of multiple SwinTransformer modules, internally alternating between local window attention and shift window mechanisms. Local window partitioning divides the entire image into several regions, significantly reducing the computational complexity of attention; the shift window strategy effectively establishes cross-regional information pathways by translating window regions between adjacent modules, enhancing the model's ability to model the local structure and contextual relationships of the image. Furthermore, Patch Merging units are introduced between layers to progressively compress the resolution of feature maps through spatial downsampling and simultaneously expand the channel dimension, thereby achieving the extraction and expression of multi-scale semantic information. This hierarchical representation provides rich and compact image feature support for subsequent joint modeling with other information sources.

[0025] S2. Based on the real-time micro-meteorological observation data collected in the transmission line scenario, the standard Transformer structure is used to extract quantitative data information sources; the standard Transformer structure is trained by constructing a micro-meteorological observation dataset for transmission lines.

[0026] The power transmission line micro-meteorological observation dataset was constructed by collecting historical micro-meteorological observation data of power transmission line scenarios. This data was obtained using different types of micro-meteorological sensors, such as those for temperature, humidity, and wind speed.

[0027] This step specifically includes: S201. The acquired micrometeorological observation data are uniformly mapped to a high-dimensional representation space to construct an initial quantitative data embedding sequence, and position information is injected into the quantitative data embedding sequence in the form of position encoding.

[0028] S202. Multiple stacked Transformer basic modules are used to extract quantitative data information sources, which are used to reflect the deep expression of the dynamic change characteristics of quantitative data.

[0029] Each Transformer basic module includes a multi-channel attention sublayer and a feedforward transformation sublayer. The multi-channel attention sublayer is used to compute multiple attention paths in parallel, capturing key associations between variables in the quantitative data embedding sequence from different subspaces, thereby enhancing the model's multivariate modeling capabilities. The feedforward transformation sublayer is used to improve the expressiveness of the representation through nonlinear mapping.

[0030] Each Transformer basic module is equipped with residual connections and layer normalization mechanisms to alleviate gradient vanishing problems in deep networks and improve the stability of training standard Transformer structures.

[0031] Ultimately, the Transformer is able to extract deep representations that reflect the dynamic changes in quantitative data, which can then be used as one of the inputs for joint modeling with image information sources.

[0032] S3. Employs a region-aware semantic fusion mechanism to perform deep semantic alignment and aggregation between image information sources and quantitative data information sources, and outputs the final features.

[0033] This invention introduces a region-aware semantic fusion mechanism to achieve deep semantic alignment and efficient aggregation between image information sources and quantitative data information sources, enhancing the complementarity and expressive power of multi-source features in weather condition recognition. This mechanism can fully mine the spatial structure information in image features and the global trend information in quantitative data features, and achieve effective interaction between information sources through channel-guided aggregation. Specifically, the image feature extraction part employs a recursive gated convolutional structure to enhance image region awareness, the quantitative data feature extraction part uses an attention mechanism to extract key channel information from the quantitative data information source, and the channel-guided aggregation part aggregates the two types of features through dynamic weighting, such as... Figure 2 As shown.

[0034] , These are the two inputs to the RSF module, representing the outputs of the 4-stage Swing Transformer and the Transformer, respectively. For By introducing recursive gated convolutions, the spatial dependencies of features can be better captured, thereby improving the representation quality of features across different images. This module first performs a dimensionality transformation, using a 1×1 convolution to... The number of channels is doubled, resulting in The data is divided into two parts, pw and pa, with different information proportions. pw contains less channel information, while pa contains more channel information. In the pa branch, depthwise convolution is used to achieve spatial feature interaction. The output features are divided according to their order. The features in the first half are weighted and aggregated with pw; the higher-order features in the second half are aggregated through cyclic interaction to achieve multi-level spatial information aggregation. Finally, a 1×1 convolution is used to restore the features to their original dimensions. This completes the entire spatial interaction process. The specific process is as follows:

[0035]

[0036]

[0037]

[0038]

[0039]

[0040] Here, `split` represents channel feature segmentation, `conv` represents convolution, and `dwconv` represents depthwise separable convolution. Indicates pixel-by-pixel multiplication. This represents a high-order spatial interactive convolution module. Through the above processing, the information interaction of the surrounding region of the residual unit features is considered in the RSF module, providing the model with a more powerful feature representation capability and further improving the performance of power line weather condition classification.

[0041] for By dynamically learning the importance of different channels in power line weather condition classification through the channel focus module, we can obtain... It aligns to through the fully connected layer The dimension, to obtain This process can be represented as

[0042]

[0043]

[0044]

[0045] Here, AvgPool and MaxPool represent average and max pooling, ReLU represents the activation function, and FC represents a fully connected layer.

[0046] The RSF module ultimately adopts a channel-guided aggregation mechanism to... and Perform aggregation and output the final features. The process is as follows:

[0047] in, This indicates channel-by-channel multiplication.

[0048] The aggregated feature map output by the RSF module contains spatial structural information of the image information source and global semantic information of the quantitative data information source.

[0049] S4. Based on the final features, the classification head is used to identify the weather condition status and output the probability distribution of the weather condition category in real time.

[0050] Specifically, the aggregated feature map is compressed into a fixed-length one-dimensional vector through global average pooling to preserve the aggregated global representation information. This feature vector is then transformed through a fully connected layer, and the probability distribution of weather condition categories is output through a Softmax function, thus completing the final multi-class classification task.

[0051] According to another aspect of the present invention, a micro-meteorological identification system for transmission lines based on multimodal data fusion is provided, comprising: The image information source extraction module is used to extract image information sources from real-time meteorological images of power transmission line scenarios using the SwinTransformer network structure; the SwinTransformer network structure is trained by constructing a meteorological visualization dataset of power transmission lines.

[0052] The quantitative data information source extraction module is used to extract quantitative data information sources based on real-time micro-meteorological observation data collected in the transmission line scenario, using a standard Transformer structure; the standard Transformer structure is trained by constructing a micro-meteorological observation dataset for transmission lines.

[0053] The aggregation module is used to perform deep semantic alignment and aggregation between image information sources and quantitative data information sources using a region-aware semantic fusion mechanism, and output the final features.

[0054] The micro-weather recognition module is used to identify weather conditions based on the final features using a classification head, and outputs the probability distribution of real-time weather condition categories.

[0055] To verify the effectiveness of the proposed regional perception semantic fusion mechanism, an effectiveness verification experiment was conducted. Using a single-source image recognition model and a single-source quantitative data sequence recognition model as benchmarks, the RSF module of this invention was introduced to construct a weather condition recognition model that aggregates image and quantitative data information sources. The performance of each model in terms of recognition accuracy, precision, recall, and F1-score was compared. The experimental results are shown in Table 1.

[0056] Table 1. Experimental Verification of the Effectiveness of the Region-Aware Semantic Fusion Mechanism

[0057] As shown in Table 1, after introducing the RSF module based on the aggregation of image information sources and quantitative data information sources, the model's recognition accuracy improved to 92.41%, a 1.4% improvement compared to using image information sources alone. It achieved the best results in all performance metrics, indicating that RSF has significant advantages in aggregating semantic information and guiding feature enhancement. In particular, the F1-score improved from 89.94% to 91.70%, demonstrating that the aggregation model significantly improved overall recognition stability and robustness while maintaining accuracy.

[0058] Thus far, the effectiveness and practicality of the regional perception semantic fusion mechanism in the task of identifying weather conditions on power lines have been verified through effectiveness experiments.

[0059] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0060] It should be noted that, depending on the implementation needs, the various steps / components described in this application can be broken down into more steps / components, or two or more steps / components or parts of the operation of steps / components can be combined into new steps / components to achieve the purpose of this invention.

[0061] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for identifying micro-meteorological conditions in transmission lines based on multimodal data fusion, characterized in that: include: The image information source is extracted from real-time meteorological images of power transmission lines using the Swing Transformer network structure. The Swin Transformer network architecture is trained by constructing a meteorological visualization dataset of power transmission lines. Based on real-time micrometeorological observation data collected in the transmission line scenario, a standard Transformer structure is used to extract quantitative data information sources; the standard Transformer structure is trained by constructing a micrometeorological observation dataset for transmission lines. The micrometeorological observation dataset was constructed by collecting micrometeorological observation data under historical conditions in power transmission line scenarios. The micrometeorological observation data was obtained through different types of micrometeorological sensors, including temperature, humidity, and wind speed. A region-aware semantic fusion mechanism is used to perform deep semantic alignment and aggregation between image information sources and quantitative data information sources, and output the final features. Based on the final features, a classification head is used to identify the weather condition status and output the probability distribution of the weather condition category in real time; The specific mechanisms of region-aware semantic fusion include: A recursive gated convolutional structure is used to transform the image information source to obtain an enhanced image information source. Attention mechanisms are used to extract key channel information from quantitative data sources; A channel-guided aggregation mechanism is adopted to aggregate enhanced image information sources and key channel information, and output the final features. The final features include the spatial structure information of the image information sources and the global semantic information of the quantitative data information sources.

2. The method for identifying micro-meteorological conditions of transmission lines based on multimodal data fusion according to claim 1, characterized in that: The classification header specifically includes: The final features are compressed into a fixed-length one-dimensional vector through global average pooling. The one-dimensional vector is transformed through a fully connected layer, and the probability distribution of the weather condition category is output through the Softmax function.

3. The method for identifying micro-meteorological conditions of transmission lines based on multimodal data fusion according to claim 1, characterized in that: The Swin Transformer network structure extracts image information sources through the following steps: Meteorological images are divided into non-overlapping patches of fixed size and converted into vector representations through linear transformation to construct an initial image embedding sequence; A layered architecture is used to extract image information sources step by step.

4. The method for identifying micro-meteorological conditions of transmission lines based on multimodal data fusion according to claim 3, characterized in that: In the layered architecture, each layer consists of multiple Swing Transformer modules, which internally alternately stack local window attention mechanisms and window shifting mechanisms; among them, The local window attention mechanism is used to divide the entire image into several regions, thereby reducing the complexity of attention calculation; the shift window mechanism is used to translate the shift window region between adjacent modules to establish cross-region information pathways.

5. The method for identifying micro-meteorological conditions of transmission lines based on multimodal data fusion according to claim 4, characterized in that: Each layer also includes a Patch Merging unit, which is used to progressively compress the resolution of the feature map through spatial downsampling and simultaneously expand the channel dimension to achieve the extraction and expression of multi-scale semantic information.

6. The method for identifying micro-meteorological conditions of transmission lines based on multimodal data fusion according to claim 1, characterized in that: The standard Transformer architecture extracts quantitative data information sources through the following steps: The acquired micrometeorological observation data are uniformly mapped to a high-dimensional representation space to construct an initial quantitative data embedding sequence, and location information is injected into the quantitative data embedding sequence in the form of location encoding. Multiple stacked Transformer base modules are used to extract quantitative data information sources, which are used to reflect the deep expression of the dynamic changes in quantitative data.

7. The method for identifying micro-meteorological conditions of transmission lines based on multimodal data fusion according to claim 6, characterized in that: Each Transformer basic module includes a multi-channel attention sublayer and a feedforward transform sublayer; among which... The multi-channel attention sublayer is used to compute multiple attention paths in parallel, capturing key correlations between variables in a quantitative data embedding sequence from different subspaces; The feedforward transformation sublayer is used to enhance the expressiveness of the representation through nonlinear mapping.

8. The method for identifying micro-meteorological conditions of transmission lines based on multimodal data fusion according to claim 6, characterized in that: Each Transformer basic module is equipped with residual connections and layer normalization mechanisms to alleviate gradient vanishing problems in deep networks and improve the stability of training standard Transformer structures.

9. A micro-meteorological identification system for transmission lines based on multimodal data fusion, characterized in that: include: The image information source extraction module is used to extract image information sources from real-time meteorological images of power transmission line scenarios using the SwinTransformer network structure. The Swin Transformer network architecture is trained by constructing a meteorological visualization dataset of power transmission lines. The quantitative data information source extraction module is used to extract quantitative data information sources based on real-time micro-meteorological observation data collected in the transmission line scenario, using a standard Transformer structure; the standard Transformer structure is trained by constructing a micro-meteorological observation dataset for transmission lines. The micrometeorological observation dataset was constructed by collecting micrometeorological observation data under historical conditions in power transmission line scenarios. The micrometeorological observation data was obtained through different types of micrometeorological sensors, including temperature, humidity, and wind speed. The aggregation module is used to perform deep semantic alignment and aggregation between image information sources and quantitative data information sources using a region-aware semantic fusion mechanism, and output the final features. The region-aware semantic fusion mechanism specifically includes: transforming the image information source using a recursive gated convolutional structure to obtain an enhanced image information source; extracting key channel information from the quantitative data information source using an attention mechanism; and aggregating the enhanced image information source and key channel information using a channel-guided aggregation mechanism to output the final feature, which includes the spatial structure information of the image information source and the global semantic information of the quantitative data information source. The micro-weather recognition module is used to identify weather conditions based on the final features using a classification head, and outputs the probability distribution of real-time weather condition categories.

Images