A weighted fusion method for importance differences of multi-source power grid survey data

Abstract

The present invention discloses a weighted fusion method for the importance difference of multi-source survey data of power grid, comprising the following steps: firstly, a multi-scale feature extraction module is used to extract feature sequences from pre-processed high-resolution remote sensing images and DEM data respectively, and then the feature sequences are input into a convolutional attention fusion module for fusion; at the same time, TextCNN is used to extract feature sequences from pre-processed geotechnical, hydrological and meteorological data respectively; then the above four feature sequences are respectively mapped to different subspaces, and the respective private and shared feature vectors are learned through encoders, and finally the shared feature vectors are input into a single-mode generation module, and the private and shared feature vectors are input into a cross-modal collaborative attention module, and a multi-modal fusion representation containing the geomorphic information of the surrounding objects of the survey area is generated through a weighted fusion strategy. The present invention fully couples surveying and mapping, geotechnical, hydrological and meteorological data, and can fully and truly present the information around the survey area, thereby ensuring the reliability of power grid line planning.

Description

Technical Field

[0001] The present invention relates to the technical field of multi-source data processing of power grids, and in particular to a heterogeneous multi-source combined navigation information processing method for unmanned aerial vehicles. Background Art

[0002] Grid line design is the key to ensuring stable supply and efficient operation of the power system. By integrating surveying and mapping data, geological and geomorphological data, and hydrological and meteorological data, multi-source data fusion technology can provide a multi-dimensional environmental consideration framework for the design of intelligent power grid lines, and enhance the comprehensive grasp of surrounding environmental factors during line design. Fusion of multi-source data for power grid line design can accurately assess the construction challenges brought by topography, predict the long-term impact of meteorological conditions on power grid stability, and avoid potential geological disaster risks, which is conducive to optimizing line path selection and ensuring long-term stable operation and efficient maintenance of the power grid. Data fusion refers to the optimization and combination of complementary and redundant information of multiple different sensors on the same target on different characteristics through certain methods, and the comprehensive analysis and calculation of multi-source data, so as to obtain a more accurate, comprehensive and consistent representation or estimation of the observed target, and improve the efficiency of data use.

[0003] The early data fusion stage mainly relied on rule-based and statistical methods. Among them, rule-based data fusion refers to integrating data from different sensors through predefined logical rules (expert system, decision logic). However, this method uses static predefined rules and is difficult to adapt to environmental changes and unknown situations; statistical methods focus on using mathematical and statistical principles to integrate information from multiple sensors or data sources. Because this method assumes that the data follows a specific distribution, it is easy to lose important information.

[0004] With the development of modern technology, data fusion algorithms have begun to integrate modern data processing technologies and concepts, and data-level fusion and feature-level fusion methods have been widely used. Data-level fusion refers to the direct fusion of the original observation data of multiple sensors at the lowest level, and then extracting feature vectors from the fused data for judgment and identification. It mainly includes Kalman filter and weighted average method. This fusion method retains more detailed information, but is more sensitive to noise and interference and requires strong error correction processing capabilities; feature-level fusion is to extract features from raw data and fuse them, such as principal component analysis (PCA) and cluster analysis. This method pays more attention to the selection and extraction of features when processing information, but it is easy to lose useful information, affecting the final decision quality.

[0005] At present, with the development and maturity of global big data multi-source information fusion technology, data fusion technology mainly focuses on decision-level fusion and model-level fusion. Decision-level fusion is based on feature or data level fusion, using various decision support algorithms to generate final decisions, such as Bayesian method, evidence theory, DS evidence theory, etc., but this method cannot fully consider all features and contextual information in the data; in order to capture the complex feature relationship in multi-source data, model-level fusion has gradually become a mainstream method, such as hidden Markov model (HMM), deep learning, etc. Fusion methods based on deep learning can integrate and learn features in multi-source data, and representative network structures include RNN, CNN and Transformer. Although the model-level fusion algorithm shows strong feature integration capabilities, its limitations are reflected in its dependence on high-quality data, uncertainty in generalization ability, and quadratic complexity of the fusion process.

[0006] The fusion of multi-source survey data of power grid is a key link to ensure the safety and stability of power engineering. It involves integrating multiple data types (including vector data, raster data, document data, etc.) from different professional fields (such as surveying, geotechnical, hydrology, meteorology, etc.). These data have the characteristics of spatiotemporal diversity and scale complexity, which are crucial for power grid planning, design and construction. At present, some scholars have proposed a convolution-based fusion network for the fusion of power grid survey data, which integrates the remote sensing spectral feature information and local spatial attributes in the survey data, analyzes ground traffic, plant coverage and other conditions, and facilitates line selection for designers.

[0007] The above methods can effectively fuse the characteristics of multi-source data. However, there is currently no algorithm specifically designed to fuse the characteristics of multi-source data for power grid survey. The existing technologies have the following main disadvantages:

[0008] (1) The data sources of existing power grid data fusion are relatively single, generally only targeting surveying and mapping data (such as remote sensing images), without comprehensively utilizing the characteristics of multi-source survey data such as geotechnical, hydrological and meteorological data. As a result, the fusion results cannot fully capture and reflect the dynamic changes and complexity of the environment around the power grid lines.

[0009] (2) Current multi-source data fusion models often ignore the importance of differential features in multimodal data when extracting features, and instead focus on common features such as building shape, river flow direction, and road direction, resulting in the inability to fully capture the complexity of the environment surrounding power grid lines.

[0010] (3) Traditional feature fusion methods usually simply splice different features together directly without considering the possible differences in the importance or contribution of each feature, and may extract some repeated features, resulting in information redundancy in the fused feature space. Summary of the invention

[0011] There are many studies on fusion algorithms based on deep learning. The Transformer model solves the problems of long-distance dependency capture and parallel computing in the field of multi-source data fusion through its self-attention mechanism. It is particularly suitable for processing sequence data containing complex interactive information. The Transformer-based multi-subspace fusion framework proposed in 2024 further solves the problems of inter-modal differentiated information fusion, redundant information reduction, and modal data heterogeneity processing in multi-source data fusion. However, there is currently no research specifically aimed at the fusion of multi-source survey data for power grids. The patent of this invention adopts a multi-subspace fusion framework based on Transformer to improve the traditional method of direct feature splicing. The purpose of the invention is to provide a weighted fusion method for the importance differences of multi-source survey data for power grids, thereby solving the aforementioned problems existing in the prior art.

[0012] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0013] A weighted fusion method for importance differences of multi-source power grid survey data includes the following steps:

[0014] S100, using the multi-scale feature extraction module to extract deep features from the pre-processed high-resolution remote sensing images and DEM data, and using TextCNN to extract deep features from geotechnical data, hydrological survey data and meteorological data; obtaining feature sequences of five types of data;

[0015] S200, using a convolutional attention feature fusion module to fuse the feature sequences of the high-resolution remote sensing image and the DEM data to obtain a feature sequence of surveying and mapping data;

[0016] S300, mapping the feature sequences of surveying and mapping data, geotechnical data, hydrological survey data and meteorological thematic data to different subspaces to learn the private representation and shared representation of the modality; through the cross-modal collaborative attention module, each modality can be simultaneously mapped with the other three modalities to reduce the heterogeneity between modalities; at the same time, using the single-modal vector generation module to fuse the private representation and shared representation of each modality to generate a single-modal vector that ultimately contains multi-faceted survey information;

[0017] S400. Through a weighted fusion strategy, in order to solve the problem of importance difference of survey data in different areas of the line, first, the importance difference coefficient of the data in a specific area is initialized; then, the coefficient is iterated according to a simulated annealing algorithm to obtain the optimal difference coefficient of the survey data in the specific area; finally, the single-modal feature vector and the cross-modal feature vector are weightedly allocated to generate a multimodal fusion vector containing the surrounding landform information of the survey area, and the multimodal fusion vector has both unified information and differentiated information within and between modes, so as to achieve comprehensive capture and reflection of the surrounding environment of the power grid line, and ensure the accuracy of power grid line planning and the reliability of design.

[0018] Preferably, the specific steps of the weighted fusion method include:

[0019] Step 1: Preprocessing the multi-source survey data of the power grid;

[0020] Step 2, high-resolution remote sensing image feature extraction: After preprocessing the multi-source survey data of the power grid according to step 1, the sliced ​​remote sensing image is obtained as the input data of the feature extraction module, and the shape, texture, and spectral feature sequences of the high-resolution remote sensing image are extracted based on the multi-scale feature extraction module;

[0021] Step 3: Extract DEM data features. According to the feature extraction method of high-resolution remote sensing images, a feature sequence including terrain features such as slope, slope aspect and terrain undulation is obtained.

[0022] Step 4: Feature extraction of geotechnical engineering survey data: The word vector output by ALBERT is fed into the TextCNN model; the TextCNN model consists of multiple convolutional layers of different sizes, pooling layers with different pooling strategies, and fully connected layers to obtain the feature sequence of geotechnical data;

[0023] Step 5: Based on the feature extraction method of geotechnical engineering survey data in step 4, the hydrological survey data and meteorological thematic data are respectively obtained through the TextCNN model to obtain feature sequences containing multi-scale features in the hydrological survey data and meteorological thematic data.

[0024] Preferably, the specific steps of the weighted fusion method also include:

[0025] Step 6: Use the multi-scale convolutional attention (MSCA) module to fuse the data feature sequences obtained in step 2 and step 3 to obtain the surveying and mapping data feature sequence;

[0026] Step 7: According to the four feature sequences obtained in Step 4, Step 5 and Step 6, use a fully connected layer to map the four feature representations to the same dimension; input the feature sequence that has undergone the differential attention mechanism into the modality private learning module to learn the difference information between different feature sequences; at the same time, input the feature sequence into the shared learning module to learn the sharing between different feature sequences;

[0027] Step 8: In the unimodal vector generation module, a fully connected layer is used to fuse the private representations and shared representations of the four modalities to generate the final unimodal vector;

[0028] Step 9: Share each modality and use the cross-modal collaborative attention module to identify the inter-modal correlation information, contextual information, complementary features, and spatiotemporal features contained in the surveying, geotechnical, hydrological, and meteorological data, model the interaction between different modalities, and obtain a multimodal fusion vector;

[0029] Step 10: The unimodal feature vector and the multimodal feature vector obtained in the cross-modal collaborative attention module are weighted fused to obtain the final multimodal fusion vector.

[0030] Preferably, the specific method of step 1 is:

[0031] First, high-resolution remote sensing images, DEM data, geotechnical engineering survey data, hydrological data and meteorological data are converted into formats, coordinate systems are unified, data are registered and resampled. This can convert data from different data sources into a unified data format, eliminate the boundaries and scale differences between the data, and unify them into the same space-time framework; secondly, the multi-source survey data of the power grid is converted into a form that meets the input requirements of the feature extraction network format.

[0032] Preferably, the specific method of step 2 is:

[0033] Firstly, the multi-scale feature extraction module is used to extract deep feature sequences of vegetation, water bodies, roads, crossings and buildings from the panchromatic image data.

[0034] Thirdly, three 3×3 convolutional layers with different dilation rates are used to extract multi-scale feature information from the feature sequence after the ReLU activation function, and the dilation rates are set to {1, 3, 5} respectively.

[0035] Preferably, the specific method of step 6 is:

[0036] The spectrum, texture, shape and color characteristics of the survey area provided by high-resolution remote sensing images are integrated with the terrain characteristics such as slope, aspect and terrain undulation provided by DEM images to obtain a surveying and mapping feature sequence including spectrum, elevation, shape and color.

[0037] The multi-scale channel attention module and the spatial attention module are combined with each other in sequence and integrated into the CNN architecture for end-to-end training to further improve the fusion performance of the model.

[0038] Preferably, the specific method of step 7 is:

[0039] First define a shared encoder Enc shared , the four initial feature sequences are represented by Enc shared After encoding, they are mapped into the same subspace;

[0040] Secondly, define the private encoder Enc for each of the four modalities enhanced by the differential attention mechanism privated , after Enc privated After encoding, the initial feature sequence of each modality is mapped into its own private subspace to capture the differentiated information specific to different modalities.

[0041] Preferably, the specific method of step 8 is:

[0042] The unimodal generation module is used to fuse the private representation and shared representation of the four feature sequences to obtain a unimodal vector.

[0043] Preferably, the specific method of step 9 is:

[0044] First, adding positional encoding to the shared representation of each modality allows the model to capture the order information of the sequence;

[0045] Secondly, self-attention is calculated for the shared features of the modalities: within each modality, the model uses the self-attention mechanism to calculate the importance of each element in the feature sequence of surveying and mapping, geotechnical, hydrological, and meteorological data; this allows the model to evaluate the degree of influence of each element in the feature sequence on other elements; and finally outputs the representation of each modality after self-attention weighted adjustment;

[0046] Next, perform cross-modal attention calculations: use the learned query, key, and value matrices to transform the feature representation of each modality; calculate the similarity between the query and all keys to obtain the attention score; use the softmax function to normalize the attention score to obtain the attention weight of each modality; after weighted summing the attention weights of each modality, aggregate the attention weights between different modalities to obtain the weighted representation of each modality on other modalities;

[0047] Finally, through the weighted fusion strategy, the unimodal feature vectors and cross-modal feature vectors are weightedly allocated to address the difference in importance of survey data in different areas of the line, and a multimodal fusion vector containing the surrounding landform information of the survey area is generated.

[0048] The beneficial effects of the present invention are as follows: the present invention discloses a weighted fusion method for importance differences of multi-source survey data of power grids. Compared with the prior art, the present invention has the following beneficial effects:

[0049] (1) Since the multi-source survey data of power grids come from different sources, with different data formats, shooting angles, expression scales, and contents, the existing data fusion methods mainly design fusion models for surveying and mapping data (such as remote sensing images). Therefore, these network architectures cannot jointly process different types of survey data. This method fully couples surveying and mapping, geotechnical, hydrological, and meteorological data, and can fully and truly present the information of landforms and geomorphology around the survey area. When facing a complex and changing external environment, it can make a more accurate and rapid response, thereby ensuring the reliability and efficiency of power grid line planning.

[0050] (2) This method uses Transformer to fuse multiple survey data such as surveying and mapping, geotechnical engineering, hydrology, and meteorology. By mapping the initial feature representation of the survey data to different subspaces, we can finally obtain unified information of each type of data and differentiated information within and between data, effectively integrating multi-dimensional features of different scales, and truly, objectively and comprehensively reflecting the vegetation, houses, roads, hydrology, meteorology, and geological information around the survey area, thereby improving the accuracy of power grid planning and the reliability of design.

[0051] (3) The surveying and mapping data provide rich surface features, while the importance of geotechnical, hydrological, and meteorological data varies in different environments. This method adopts a weighted feature fusion method that can fully consider the differences in the importance of different features. The allocation of weight coefficients reflects the comprehensive contribution and importance of each data source to the power grid survey task, ensuring that the fusion results are both comprehensive and targeted, meeting the specific needs of power grid planning and design. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 It is the technical roadmap of the present invention;

[0053] Figure 2 is a schematic diagram of the MSCA attention module of the present invention;

[0054] Figure 3 It is a structural schematic diagram of a cross-modal collaborative attention module of the present invention taking survey data as an example;

[0055] Figure 4 Schematic diagram of the ALBERT model structure using geotechnical data as an example in the present invention;

[0056] Figure 5 This is the process of the simulated annealing algorithm of the difference weighted coefficient in the present invention, taking the crossing river area as an example;

[0057] Figure 6 The present invention is a flow chart of a weighted fusion method for importance differences of multi-source survey data of power grids. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation methods described herein are only used to explain the present invention and are not used to limit the present invention.

[0059] In the early planning stage of power grid construction, detailed survey of the surrounding environment of the tower base is crucial. The traditional survey process relies on manual on-site mapping and investigation, but it is often inefficient due to inconvenient transportation, harsh environment and extreme weather. With the advancement of remote sensing technology and space science, it is now possible to efficiently collect key topographic, hydrological and meteorological information over a large area through satellite remote sensing and professional equipment. High-resolution remote sensing images provide a macroscopic view of the topography including roads, vegetation, buildings, bridges and important crossings. Digital elevation model (DEM) data records the elevation changes of the terrain in a rasterized form, which helps to analyze the slope, slope aspect and terrain undulation. Geotechnical engineering survey data records key information such as soil composition, sand and gravel percentage and soil viscosity at a specific location in text or spreadsheet form. Hydrological data records information such as river depth, flow rate and basin length in detail. In addition, meteorological data covers key meteorological conditions such as temperature and wind speed. The comprehensive analysis of these multi-source data is of great significance for the accurate planning and efficient construction of power grid projects.

[0060] In order to present the geomorphic information and real environment of the surrounding areas of the survey area more comprehensively and clearly through complementary information, and to provide auxiliary decision-making basis for the site selection and line selection of power grid survey, it is necessary to propose how to integrate the above multi-source survey data through information technology.

[0061] There are many studies on fusion algorithms based on deep learning. The Transformer model solves the problems of long-distance dependency capture and parallel computing in the field of multi-source data fusion through its self-attention mechanism. It is particularly suitable for processing sequence data containing complex interactive information. The Transformer-based multi-subspace fusion framework proposed in 2024 further solves the problems of inter-modal differentiated information fusion, redundant information reduction, and modal data heterogeneity processing in multi-source data fusion. However, there is currently no research specifically aimed at the fusion of multi-source survey data for power grids. The patent of this invention adopts a multi-subspace fusion framework based on Transformer, improves the traditional method of direct feature splicing, and proposes a weighted fusion strategy for the importance differences of multi-source survey data for power grids.

[0062] Reference Figure 1 , Figure 2 , Figure 3 , Figure 4 , Figure 5 and Figure 6 A weighted fusion method for importance differences of multi-source survey data of power grid is shown, comprising the following steps:

[0063] S100, use the multi-scale feature extraction module to extract deep feature sequences from the pre-processed high-resolution remote sensing images and DEM data, and use TextCNN to extract deep feature sequences from the pre-processed geotechnical data, hydrological survey data and meteorological thematic data.

[0064] S200, using the convolutional attention feature fusion module to fuse the feature sequences of the high-resolution remote sensing image and the DEM data to obtain the feature sequence of the surveying and mapping data.

[0065] S300, map the feature sequences of surveying and mapping data, geotechnical data, hydrological survey data and meteorological thematic data to different subspaces to learn the private representation and shared representation of the four modalities of surveying and mapping, geotechnical data, hydrological survey data and meteorological data; through the cross-modal collaborative attention module, each modality can be simultaneously mapped with the other three modalities to reduce the heterogeneity between modalities; at the same time, use the three-modal representation generation module to fuse the private representation and shared representation of each modality to generate a single modal vector that ultimately contains multi-faceted survey information.

[0066] S400. Through the weighted fusion strategy, in order to solve the problem of importance difference of survey data in different areas of the line, first, the importance difference coefficient of data in a specific area is initialized. Then, the coefficient is iterated according to the simulated annealing algorithm to obtain the optimal difference coefficient of survey data in a specific area. Finally, the single-modal feature vector and the cross-modal feature vector are weightedly allocated to generate a multimodal fusion vector containing the surrounding landform information of the survey area. The multimodal fusion vector has both uniform information and intra-modal and inter-modal differentiation information, so as to achieve comprehensive capture and reflection of the surrounding environment of the power grid line, and ensure the accuracy of power grid line planning and the reliability of design.

[0067] Preferably, the specific steps of the weighted fusion method include:

[0068] Step 1: Preprocess the multi-source power grid survey data.

[0069] Step 2, high-resolution remote sensing image feature extraction: After preprocessing the multi-source survey data of the power grid according to step 1, the sliced ​​remote sensing image is obtained as the input data of the feature extraction module, and the shape, texture, and spectral feature sequences of the high-resolution remote sensing image are extracted based on the multi-scale feature extraction module.

[0070] Step 3: Extract DEM data features. Based on the feature extraction method of high-resolution remote sensing images, a feature sequence containing terrain features such as slope, slope aspect and terrain undulation is obtained.

[0071] Step 4: Feature extraction of geotechnical engineering survey data: The word vector output by ALBERT is fed into the TextCNN model; the TextCNN model consists of multiple convolutional layers of different sizes, pooling layers with different pooling strategies, and fully connected layers to obtain the feature sequence of geotechnical data.

[0072] TextCNN selects the convolution kernels of the three branches as {2×2, 3×3, 4×4}. At the same time, the convolution kernels of different sizes are adjusted during convolution and different pooling strategies are adopted. After the convolution operation with a convolution kernel size of 2, the model adopts the Maxpooling method to divide the feature map output after convolution into several small blocks of the same size without overlapping. Only the largest eigenvalue is taken in each small block, and finally a new feature map is generated, ignoring the information in the geotechnical data that is not important to the survey area, and focusing on the keywords such as geotechnical type, viscosity, and sand content; after the convolution operation with a convolution kernel size of 3, the K-Maxpooling method is adopted. This method believes that not only the maximum value point in each pooling block is important, but also the first few bright spots are more important, so the first k largest values ​​are taken in each pooling block to obtain a new feature map; in the convolution operation with a convolution kernel size of 4, the model adopts an average pooling strategy to consider the information of each word in the geotechnical data. Finally, the feature mapping matrices of the three branches are spliced ​​through the fully connected layer to obtain the final feature sequence containing multi-scale features.

[0073] Step 5: Based on the feature extraction method of geotechnical engineering survey data in step 4, the hydrological survey data and meteorological thematic data are respectively obtained through the TextCNN model to obtain feature sequences containing multi-scale features in the hydrological survey data and meteorological thematic data.

[0074] Preferably, the specific steps of the weighted fusion method also include:

[0075] Step 6: Use the multi-scale convolutional attention (MSCA) module to fuse the feature sequences obtained in step 2 and step 3 to obtain the surveying and mapping data feature sequence;

[0076] Step 7. According to the four feature sequences obtained in steps 4, 5 and 6, use a fully connected layer to map the four feature representations to the same dimension; input the feature sequence that has undergone the differential attention mechanism into the modality private learning module to learn the difference information between different feature sequences; at the same time, input the feature sequence into the shared learning module to learn the sharing between different feature sequences.

[0077] Step 8: In the unimodal vector generation module, the private representations and shared representations of the four modalities are fused through a fully connected layer to generate the final unimodal vector;

[0078] Step 9: Share each modality and use the cross-modal collaborative attention module to identify the inter-modal correlation information, contextual information, complementary features, and spatiotemporal features contained in the surveying, geotechnical, hydrological, and meteorological data, model the interaction between different modalities, and obtain a multimodal fusion vector;

[0079] Step 10: The unimodal feature vector and the multimodal feature vector obtained in the cross-modal collaborative attention module are weighted fused to obtain the final multimodal fusion vector.

[0080] Preferably, the specific method of step 1 is:

[0081] First, high-resolution remote sensing images, DEM data, geotechnical engineering survey data, hydrological data and meteorological data are converted into formats, coordinate systems are unified, data registration and data resampling are performed, so that data from different data sources can be converted into a unified data format, and the boundaries and scale differences between data can be eliminated and unified into the same space-time framework. Secondly, the multi-source survey data of the power grid is converted into a form that meets the input requirements of the feature extraction network format. The specific process is as follows:

[0082] (1) Generate vector representation of geotechnical engineering survey data. First, the geotechnical data is cleaned and irrelevant characters and numbers are removed to reduce noise interference. Input to the ALBERT pre-trained model. The input geotechnical data is segmented by a tokenizer, and a series of tags (Tokens) are output with [start] and [end] at the beginning and end respectively. Then, each input Token is converted into a word embedding vector through word embedding. Secondly, the word embedding vector is passed through the ALBERT pre-trained language model to output the E of each Token. n Feature sequences. These feature sequences capture the deep semantic information of the input geotechnical data and facilitate the input into the subsequent feature extraction network.

[0083] (2) Hydrological survey data and meteorological data are processed in the same way as geotechnical data to generate the E required for model input. n Feature sequence.

[0084] (3) Slice the high-resolution remote sensing image. First, the survey area is cropped and the image resolution is adjusted to match the network input size. Then, to enhance the generalization ability of the model, these slices are enhanced, such as rotation, scaling, or flipping. Finally, these slices are converted into a format that the neural network can process, completing the complete adaptation process from the original remote sensing image to the network input.

[0085] (4) The DEM data is also sliced ​​and converted into a format that can be processed by the neural network according to the remote sensing image method.

[0086] Preferably, the specific method of step 2 is:

[0087] High-resolution remote sensing image feature extraction: After the above preprocessing, the sliced ​​remote sensing image is used as the input data of the feature extraction module. The shape, texture and spectral feature sequences of the high-resolution remote sensing image are extracted based on the multi-scale feature extraction module.

[0088] Firstly, the multi-scale feature extraction module is used to extract deep feature sequences of vegetation, water bodies, roads, crossings and buildings from the panchromatic image data.

[0089] A three-branch convolution layer is used, with 1×1, 3×3, and 5×5 convolution kernels to compare the input high-resolution remote sensing image X of the survey area. image ∈R p×p×k Convolution operation is performed, and each channel is multiplied point by point and summed to capture local features such as spectrum, color, texture, edge, etc., which are used to distinguish different objects (houses, trees, roads, important crossings) and identify edge information, and realize linear transformation between channels; secondly, the features output by the convolution layer are standardized through batch normalization (BatchNormalization, BN), so that the pixel value data distribution of each feature channel is close to the standard normal distribution, such as formula (1) ~ (2), which helps to improve the stability of network training.

[0090]

[0091] Where x = [x1, x2, ... x n ], x i Represents the pixel value corresponding to a sample feature channel, n is the number of samples in the batch, is the result of standardized output, is the final result of the BN layer, μ is the mean of X, σ 2 is the variance of X, ε is a small constant, γ and β are learnable parameters, which are used to scale and translate the normalized data, respectively, γ is the scaling factor, and β is the offset.

[0092] Thirdly, three 3×3 convolutional layers with different dilation rates are used to extract multi-scale feature information from the feature sequence after the ReLU activation function, and the dilation rates are set to {1, 3, 5} respectively.

[0093] By introducing a dilation rate during the convolution process, the receptive field can be increased while maintaining computational efficiency, allowing the network to better capture contextual information in the image. Each convolutional layer outputs a feature sequence that reflects features at different scales. Finally, a 1×1 convolutional layer is used to integrate and obtain a single feature sequence containing all feature maps, thereby fusing deep data features at different scales (such as spectral, texture, shape, and spatial context features of roads, buildings, and vegetation in the survey area).

[0094] Preferably, the specific method of step 6 is:

[0095] The data features obtained in step 2 and step 3 are fused using the multi-scale convolutional attention (MSCA) module.

[0096] The spectrum, texture, shape and color features of the survey area provided by high-resolution remote sensing images are fused with terrain features such as slope, aspect and terrain undulation provided by DEM images to obtain a fused mapping feature sequence containing spectrum, elevation, shape and color. Global channel attention captures overall features through global pooling, while local channel attention focuses on more detailed features.

[0097] The multi-scale channel attention module and the spatial attention module are combined with each other in sequence and integrated into the CNN architecture for end-to-end training to further improve the fusion performance of the model.

[0098] For power grid survey, roads, rivers, forest areas, etc. have higher semantic significance. Under specific terrain conditions, the distribution of vegetation and houses has a greater impact on the stability of the power grid because they directly affect the layout and construction of the power grid.

[0099] Step 6-1: The multi-scale channel attention module extracts global and local channel attention weights by using two branches with different scales.

[0100] First, the global branch is used to map the feature map F = [f1, ..., f c ]∈R H×W×C Each channel of is subjected to global maximum pooling and global average pooling respectively, thereby obtaining two global channel features of dimension 1×1×C and Where C represents the number of feature map channels, H represents the feature map height, and W represents the feature map width. Here, 1,1 means that the height and width of the feature map become 1 after the pooling operation. Then, the two global channel features are respectively input into a shared fully connected neural network consisting of a multi-layer perceptron (MLP) and a hidden layer and processed. MLP can learn the importance of different channel features and generate an attention weight map, which can be used to modulate the original feature map so that the network can pay more attention to the most critical features for the power grid survey task. Finally, the two processed feature maps are added together and activated by the Sigmoid function to obtain the global channel attention map M. CG (F).

[0101] The above global channel attention module workflow can be summarized as formula (6):

[0102]

[0103] Among them, Maxpool, Avgpool, MLP are global maximum pooling, global average pooling and multi-layer perceptron respectively, σ, W0, W1 are the weights of activation function and multi-layer perceptron respectively.

[0104] At the same time, using local branches F = [f1, ..., f c ]∈R H×W×C The feature map is convolved with 1×1 to reduce the dimension and change the channel dimension. Then, the reduced feature map is mapped back to the original number of channels using 1×1 convolution, and finally activated by the Sigmoid function. The local channel attention map M is obtained. CL (F), formulas as (7) to (9).

[0105] F′=F*K (7)

[0106] F″=F′*K′ (8)

[0107] M CL (F) = σ(F″) (9)

[0108] Where * represents the convolution operation. K is a 1x1 convolution kernel used to reduce the number of channels. K′ is a 1x1 convolution kernel used to restore the number of channels. σ is the Sigmoid function, and A is the attention weight map obtained after Sigmoid activation. M CL (F) is the local channel attention map.

[0109] Then, the global channel attention map M obtained above is CG (F) and local channel attention map M CL (F) is multiplied by the corresponding elements of the feature map F to obtain the global feature map F G and local feature map F L Finally, the channel attentions of different scales are added and fused to obtain the channel attention feature map F′.

[0110] Step 6-2: The spatial attention module is used to identify the importance differences of different regions in high-resolution remote sensing images and DEM feature maps.

[0111] First, F′ is subjected to global maximum pooling and global average pooling operations based on the channel to obtain F m ' ax and F a ' vg , and concatenate these two feature maps.

[0112] Secondly, the superimposed feature map is convolved using a 7×7 convolution kernel.

[0113] Again, after activation by the Sigmoid function, the spatial attention feature map M is generated s(F′).

[0114]

[0115] Where Maxpool and Avgpool are global maximum pooling and global average pooling respectively, and σ represents the Sigmoid function. 7×7 (·) represents the convolution layer operation with a 7×7 convolution kernel.

[0116] Finally, by adding the spatial attention feature map M s The final feature map F″ is obtained by multiplying the corresponding elements of (F′) and the channel attention feature map F′.

[0117] The entire MSCA process is equivalent to formula (11):

[0118]

[0119] Among them, M CG (F) is the global channel attention map, M CL (F) is the local channel attention map, Represents element-by-element multiplication, M s (F′) is the spatial attention feature map.

[0120] Step 6-3: The remote sensing image feature map F″ obtained by the MSCA module image and DEM feature map F″ Dem , the two features are fused by element-by-element addition.

[0121] Specifically, the fusion process in the module represents the features learned from the aerial remote sensing image and DEM data as F″ image and F″ Dem , and use the element-by-element addition method to fuse the two features, as shown in formula (12):

[0122] Then, the fusion features of the obtained surveying and mapping data are

[0123] Preferably, the specific method of step 7 is:

[0124] Based on the above obtained F r 、F h 、F w , use the fully connected layer to map the four feature sequences to the same dimension. Through the modality private and shared representation learning module, learn the shared and different information between different feature sequences. Common information such as geographic location information, time information, and terrain features serve as a bridge between different data sources.

[0125] Specifically, first define a shared encoder Enc shared , the four feature sequences are represented by Enc shared After encoding, it is mapped to the same subspace; as shown in formula (13):

[0126]

[0127] Where: represents the shared representation of each feature sequence; T m represents the length of each feature sequence, d map is the feature dimension of the shared representation. m Represents their respective initial feature sequences. A simple fully connected layer is used to map the initial feature sequences of each modality, and the shared parameter of the four modalities is θ shared .

[0128] At the same time, the four feature sequences are input into the difference attention mechanism. The difference information is obtained by subtracting each two feature sequences. Then, they are passed through an average pooling layer and a sigmoid activation function layer to obtain the global difference weights of each two modalities. Finally, the residual connection is performed with the initial feature sequence to generate four feature sequences with enhanced difference information, and their respective private encoders Enc are defined for them. privated , after Enc privated After encoding, the initial feature sequence of each modality is mapped into its own private subspace to capture the differentiated information specific to different modalities.

[0129] For example, the texture, shape and color in the feature sequence of surveying and mapping data; the geological structure and soil type in the feature sequence of geotechnical data; the hydrological information in the hydrological feature sequence; and the meteorological information in the meteorological feature sequence. The process is shown in formula (14):

[0130]

[0131] In the formula Represents the difference between each modality to enhance the private representation; similarly, a fully connected layer is used to map the initial feature sequence of each modality to its own private subspace, and each modality has its own encoder with different network parameters θ privated .

[0132] Preferably, the specific method of step 8 is:

[0133] Firstly, the unimodal generation module is used to fuse the private and shared representations of the four modalities to obtain a unimodal vector.

[0134] In order to comprehensively represent the representation of a single modality, the two different representations of the same modality (private representation and shared representation) are fused through a fully connected layer to generate the final single modality vector. The output of each single modality vector is defined by formula (15):

[0135]

[0136] Where U m ∈R d , U m Represents the final single-mode vector of each mode. FC(·) represents the fully connected layer operation. σ represents the Sigmoid function. At this time, the final single-mode vector obtained integrates the information related to the survey from different aspects of the same mode. It contains both uniform information such as coverage and unique color and texture information specific to a certain mode. Preferably, the specific method of step 9 is:

[0137] First, adding positional encodings to the shared representation of each modality allows the model to capture the sequential order information.

[0138] As shown in formula (17):

[0139]

[0140] In the formula For each modality, the position code is The position-encoded representation is obtained for each modality and used as input into the subsequent network model for multimodal fusion.

[0141] Secondly, self-attention is calculated for the shared features of the modalities: within each modality, the model uses the self-attention mechanism to calculate the importance of each element in the feature sequence of surveying and mapping, geotechnical, hydrological, and meteorological data; so that the model can evaluate the influence of each element in the feature sequence on other elements; finally, the feature vector adjusted by self-attention weighting is output within each modality;

[0142] Next, perform cross-modal attention calculation: use the learned query, key, and value matrices to transform the feature vector of each modality; calculate the similarity between the query and all keys to obtain the attention score; use the softmax function to normalize the attention score to obtain the attention weight of each modality; after weighted summing the attention weights of each modality, aggregate the attention weights between different modalities to obtain the weighted feature vector of each modality representing other modalities;

[0143] Finally, the aggregated cross-modal weighted feature vector and unimodal weighted feature vector are weightedly summed to obtain a representation that integrates cross-modal information.

[0144] Specifically, we take the collaborative attention cross-modal Transformer based on the feature sequence of surveying and mapping data as an example. It contains L layers of collaborative attention cross-modal Transformer layers (abbreviated as CACT layers). For the i-th CACT layer (i=1,2,…,L), for X r , X h , X w The input of first uses the self-attention mechanism to explore the interaction of each element (texture feature, shape feature, spectral characteristic, spatial distribution feature) in each feature sequence, as shown in equations (18) to (20):

[0145]

[0146] In the formula, Query key value are modality-specific attention matrices respectively; LN(·) represents layer normalization; As input of geotechnical, hydrological and meteorological data in the next CACT layer.

[0147] After the four modalities of surveying and mapping, geotechnical, hydrological, and meteorological data interact within the modality through self-attention calculation, the interaction between surveying and mapping data and geotechnical, hydrological, and meteorological data, and the interaction between geotechnical data and surveying and mapping, hydrological, and meteorological data, and so on, each modality interacts with other modalities respectively.

[0148] Taking the cross-modal collaborative attention module based on remote sensing images as an example, the feature representation of the surveying and mapping data is projected as a query, and the representation of the geotechnical, hydrological and meteorological data is spliced ​​as the intermediate value. The intermediate value is projected to obtain the key and value. At this time, the similarity relationship between the geological related elements and the geotechnical data elements in the feature sequence of the surveying and mapping data can be obtained. Secondly, the output of the cross-modal attention is sent to the feedforward neural network to obtain the output of the cross-modal Transformer layer of the collaborative attention. As shown in equations (21) and (22):

[0149] Where: Query key value They are the cross-modal attention projection matrices respectively. is the output of the i-th layer of the collaborative attention cross-modal Transformer based on the mapping data, The vector representation of the last time step is used as the multimodal fusion vector based on surveying and mapping data. For the collaborative attention cross-modal Transformer based on geotechnical data, the same logic can be used to obtain The vector representation of the last time step is used as the final output of the collaborative attention cross-modal Transformer based on geotechnical data.

[0150] Step 10: The unimodal feature vector and the multimodal feature vector obtained in the cross-modal collaborative attention module are weighted fused to obtain the final multimodal fusion vector.

[0151] First, the importance difference coefficient of the data in a specific area is initialized. Then, the coefficient is iterated according to the simulated annealing algorithm to obtain the optimal difference coefficient of the survey data in a specific area. Finally, the single-modal feature vector and the cross-modal feature vector are weighted and allocated to generate a multimodal fusion vector containing the surrounding landform information of the survey area.

[0152] The expression of weighted feature fusion is (23):

[0153]

[0154] in, is the Kronecker product; α1, α2 are weighting coefficients; U m , is the unimodal feature vector and the multimodal feature vector; F final is the fused feature; the optimal values ​​of α1 and α2 are obtained by the simulated annealing algorithm. The simulated annealing algorithm is a simulation of the solid annealing process, with α = α1, α2 as the state vector of the simulated annealing algorithm; the energy function of the simulated annealing algorithm, that is, the loss function, is set to L(α). When the temperature is T, the probability of transferring from state α1, α2 to the new state α1′, α2′ is obtained by the following Markov chain criterion:

[0155] According to the Markov chain theory, as T→0, the random process reaches equilibrium at each temperature, L(α1, α2)→L min , at this time the optimal weighting coefficient is obtained In summary, compared with the prior art, the advantages of the present invention are as follows:

[0156] (1) Since the multi-source survey data of power grids come from different sources, with different data formats, shooting angles, expression scales, and contents, the existing data fusion methods mainly design fusion models for surveying and mapping data (such as remote sensing images). Therefore, these network architectures cannot jointly process different types of survey data. This method fully couples surveying and mapping, geotechnical, hydrological, and meteorological data, and can fully and truly present the information of landforms and geomorphology around the survey area. When facing a complex and changing external environment, it can make a more accurate and rapid response, thereby ensuring the reliability and efficiency of power grid line planning.

[0157] (2) This method uses Transformer to fuse multiple survey data such as surveying and mapping, geotechnical engineering, hydrology, and meteorology. By mapping the initial feature representation of the survey data to different subspaces, we can finally obtain unified information of each type of data and differentiated information within and between data, effectively integrating multi-dimensional features of different scales, and truly, objectively and comprehensively reflecting the vegetation, houses, roads, hydrology, meteorology, and geological information around the survey area, thereby improving the accuracy of power grid planning and the reliability of design.

[0158] (3) The surveying and mapping data provide rich surface features. The importance of geotechnical, hydrological and meteorological data varies in different environments. This method adopts a differential weighted feature fusion method, which can fully consider the differences in the importance of different line environment survey data. The allocation of weight coefficients reflects the comprehensive contribution and importance of each data source to the power grid survey task, ensuring that the fusion result is both comprehensive and targeted, meeting the specific needs of power grid planning and design.

Claims

1. A weighted fusion method for importance differences of multi-source power grid survey data, characterized in that: The following steps are involved: S100, using the multi-scale feature extraction module to extract deep features from the pre-processed high-resolution remote sensing images and DEM data of the survey area, and using TextCNN to extract deep features from geotechnical data, hydrological survey data and meteorological thematic data; obtaining feature sequences of five types of survey data; S200, using a convolutional attention feature fusion module to fuse the texture and shape features of the high-resolution remote sensing image and the elevation features of the DEM data to form a surveying and mapping data feature sequence; S300, mapping the characteristic sequences of the surveying and mapping data, the geotechnical data, the hydrological survey data and the meteorological thematic data to different subspaces to learn the private representation and shared representation of the four modalities of surveying and mapping, geotechnical data, hydrological data and meteorological data; inputting the shared representation of each modality into the cross-modal collaborative attention module to obtain a cross-modal feature vector containing information of the other three modalities, thereby reducing the heterogeneity between modalities; at the same time, inputting the private representation and shared representation of each modality into the single-modal generation module to generate single-modal feature vectors containing private and shared information respectively; S400. Through a weighted fusion strategy, in order to solve the problem of importance difference of survey data in different areas of the line, first, the importance difference coefficient of the data in a specific area is initialized; then, the coefficient is iterated according to a simulated annealing algorithm to obtain the optimal difference coefficient of the survey data in the specific area; finally, a weighted allocation is performed on the single-modal feature vector and the cross-modal feature vector to generate a multi-modal fusion vector containing the surrounding landform information of the survey area, and the multi-modal fusion vector has both uniform information and intra-modal and inter-modal differentiation information, so as to achieve a comprehensive capture and reflection of the surrounding environment of the power grid line, and ensure the accuracy of the power grid line planning and the reliability of the design; The specific steps of the weighted fusion method include: Step 1: Preprocessing the multi-source survey data of the power grid; Step 2, high-resolution remote sensing image feature extraction: After preprocessing the multi-source survey data of the power grid according to step 1, the sliced ​​remote sensing image is obtained as the input data of the feature extraction module, and the shape, texture, and spectral feature sequences of the high-resolution remote sensing image are extracted based on the multi-scale feature extraction module; Step 3: Extracting features from DEM data, obtaining a feature sequence of terrain features including slope, aspect and terrain undulation according to the feature extraction method of the high-resolution remote sensing image; Step 4, feature extraction of geotechnical engineering survey data: the word vector output by ALBERT is fed into the TextCNN model; the TextCNN model consists of multiple convolutional layers of different sizes, pooling layers with different pooling strategies, and fully connected layers to obtain the feature sequence of geotechnical data; Step 5: The hydrological survey data and the meteorological thematic data are respectively subjected to the TextCNN model according to the feature extraction method of the geotechnical engineering survey data in step 4 to obtain feature sequences containing multi-scale features in the hydrological survey data and the meteorological thematic data; The specific steps of the weighted fusion method also include: Step 6: using the multi-scale feature extraction module to fuse the data features obtained in step 2 and step 3 to obtain a surveying and mapping data feature sequence; Step 7, according to the four feature sequences obtained in step 4, step 5 and step 6, first, use the fully connected layer to map the four feature sequences to the same dimension; again, input the feature representations of the surveying, hydrology, geotechnical, and meteorological data of the projection subspace into the difference attention module, calculate the difference between each two modalities, and obtain the global difference weight after passing through an average pooling layer and a sigmoid activation function layer. After generating the difference information, the four feature representations after difference enhancement are obtained through residual connection; secondly, the private representation and shared representation of different feature sequences are obtained through the modality private and shared representation learning module; Step 8: Input the private representation and the shared representation of the same modality into a single-modality vector generation module through the fully connected layer to generate a single-modality vector that ultimately includes the private representation and the shared representation of each modality; Step 9: Share each modality and use the cross-modal collaborative attention module to identify the inter-modal correlation information, contextual information, complementary features, and spatiotemporal features contained in the surveying, geotechnical, hydrological, and meteorological data, model the interaction between different modalities, and obtain a multimodal fusion vector; Step 10: The unimodal feature vector and the multimodal feature vector obtained in the cross-modal collaborative attention module are weighted fused to obtain the final multimodal fusion representation.

2. The weighted fusion method for importance differences of multi-source power grid survey data according to claim 1 is characterized in that: The specific method of step 1 is: Firstly, the high-resolution remote sensing images, the DEM data, the geotechnical engineering survey data, the hydrological data and the meteorological data are subjected to format conversion, coordinate system unification, data registration and data resampling processing, so as to convert data from different sources into a unified data format, eliminate the boundary and scale differences between the data, and unify them into the same space-time framework; secondly, the multi-source survey data of the power grid is converted into a form that meets the input requirements of the feature extraction network format.

3. The weighted fusion method for importance differences of multi-source power grid survey data according to claim 2 is characterized in that: The specific method of step 2 is: Firstly, the multi-scale feature extraction module is used to extract shallow feature sequences of vegetation, water bodies, roads, crossings and buildings from the panchromatic image data. Thirdly, the multi-scale deep feature information is extracted by using three 3×3 convolutional layers with different dilation rates to represent the feature sequence after the ReLU activation function. The dilation rates are set to {1, 3, 5} respectively.

4. The weighted fusion method for importance differences of multi-source power grid survey data according to claim 3 is characterized in that: The specific method of step 6 is: The spectrum, texture, shape and color characteristics of the survey area provided by high-resolution remote sensing images are integrated with the terrain characteristics of slope, aspect and terrain undulation provided by DEM images to obtain a surveying and mapping feature sequence containing spectrum, elevation, shape and color. The multi-scale channel attention module and the spatial attention module are combined with each other in sequence and integrated into the CNN architecture for end-to-end training to further improve the fusion performance of the model.

5. The weighted fusion method for importance differences of multi-source power grid survey data according to claim 4 is characterized in that: The specific method of step 7 is: First define a shared encoder Enc shared , the four initial feature sequences are represented by Enc shared After encoding, they are mapped into the same subspace; Secondly, define the private encoder Enc for the four modes of surveying and mapping, geotechnical, hydrological, and meteorological privated , the four feature sequences obtained by the difference attention mechanism are enhanced by Enc privated After encoding, they are mapped into their respective private subspaces to capture differentiated information specific to different modalities.

6. The weighted fusion method for importance differences of multi-source power grid survey data according to claim 5 is characterized in that: The specific method of step 8 is: The unimodal generation module is used to fuse the private representations and shared representations of the four modalities to obtain a unimodal vector.

7. The weighted fusion method for importance differences of multi-source power grid survey data according to claim 6 is characterized in that: The specific method of step 9 is: First, adding positional encoding to the shared representation of each modality allows the model to capture the order information of the sequence; Secondly, self-attention is calculated for the shared features of the modalities: within each modality, the model uses the self-attention mechanism to calculate the attention weights of each element in the feature sequence of surveying and mapping, geotechnical, hydrological, and meteorological data; this allows the model to evaluate the degree of influence of each element in the feature sequence on other elements; and finally outputs the feature vector adjusted by self-attention weighting within each modality; Next, perform cross-modal attention calculations: use the learned query, key, and value matrices to transform the feature representation of each modality; Calculate the similarity between the query and all keys, get the attention weight between each element, and then divide it by the corresponding feature dimension d k , ensuring the gradient stability of softmax; finally, multiply the attention weight and the V matrix; use the softmax function to normalize the attention score to obtain the attention weight of each modality; After weighted summing of the attention weights of each modality, the attention weights between different modalities are aggregated to obtain the weighted feature vector of each modality representing other modalities; Finally, the aggregated cross-modal weighted feature vector, unimodal weighted feature vector and the optimal difference coefficient of the specific area survey data obtained according to the simulated annealing algorithm are weightedly summed to obtain a cross-modal information fusion representation containing importance differences.

Images