A power grid building land classification method fusing attention and multi-level CNNs

Abstract

The application discloses an electric power grid building land classification method fusing attention and multi-level CNN in the technical field of remote sensing image pixel-level classification, and comprises the following steps: step one: sending a multispectral pixel sequence to the FC, adjusting the dimension of the 1D-spectrum sequence by using the linear transformation function of the full connection layer, and reshaping the transformed 1D-spectrum sequence into a 3D-spectrum feature matrix; step two: extracting three different levels of features of the 2D-CNN as part of the input of the Transformer; step three: combining the features extracted by the CNN and the position coding together with the pixel information while adding the position coding; step four: realizing feature fusion through the skip link between adjacent Transformer Encoders; compared with the Transformer, the model fusing the Transformer and the CNN has an overall precision and Kappa value rising by 1.22% and 0.0144 respectively; the application can realize classification of the two-dimensional convolutional neural network (2D-CNN) and pixel-level classification of remote sensing images.

Description

Technical Field

[0001] This invention relates to the field of pixel-level classification technology of remote sensing images using artificial intelligence, specifically a method for classifying power grid building land use by integrating attention and multi-level CNN. Background Technology

[0002] With the continuous development of the social economy, national electricity consumption has gradually increased. To address grid-related power shortages and better serve local economies, various regions across the country have increased power grid infrastructure construction. However, lagging power grid planning and inadequate planning coordination have created significant difficulties in the construction of many major power infrastructure projects, becoming a major factor restricting power grid development. The construction of power infrastructure is often guided by specialized power grid plans prepared by the power sector, with local governments following these plans. A well-planned and appropriate site selection is of great importance. This requires the assistance of remote sensing technology, which fully considers land cover classification and topography to assist professionals in the forward-looking planning and control of land use for power construction.

[0003] Remote sensing imagery differs from standard RGB imagery. Limited by mixed pixels, complex scenes, and its own spatial resolution, it suffers from severe metaspectral distortion of similar objects. Furthermore, unlike buildings, crops, farmland, and vegetation are affected by seasonal changes, exhibiting significant differences in characteristics across different regions and time periods, leading to identification difficulties. This necessitates a comprehensive analysis of spatial, spectral, and temporal information in remote sensing imagery. However, traditional remote sensing classification algorithms, such as SVM, maximum likelihood estimation, and neural networks, do not fully utilize the spatial, spectral, and temporal information in remote sensing images, or only utilize one type of information. Currently, deep learning technology is developing rapidly, with convolutional neural networks, fully convolutional neural networks, and Transformer networks for processing temporal sequences showing advantages in image recognition. However, the creation of deep learning datasets, the interpretability of network design, and how to improve real-time performance to meet our practical applications still require further exploration and discovery. Utilizing deep learning technology for high-precision, rapid, and automated remote sensing interpretation, thereby assisting the power sector in completing power construction land planning, is of great significance.

[0004] To address this, we propose a method for classifying power grid building land use that integrates attention and multi-level CNN. Summary of the Invention

[0005] The purpose of this invention is to provide a method for classifying power grid building land use that integrates attention and multi-level CNN, so as to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a method for classifying power grid building land using a fusion of attention and multi-level CNN, comprising the following steps:

[0007] Step 1: Send a multispectral pixel sequence to the FC, use the linear transformation function of the fully connected layer to adjust the dimension of the 1D-spectral sequence, and reshape the transformed 1D-spectral sequence into a 3D-spectral feature matrix;

[0008] Step 2: Extract features from three different levels of the 2D-CNN as part of the Transformer input;

[0009] Step 3: While adding positional encoding, combine the features extracted by the CNN with the positional encoding and pixel information;

[0010] Step 4: Achieve feature fusion by skipping links between adjacent Transformer Encoders;

[0011] Step 5: Finally, the MLP Head obtains the classification results, achieving pixel-level classification;

[0012] Step 6: Classify the study area using different methods.

[0013] Preferably, step six involves classifying the study area using different methods, including the following steps.

[0014] S1: Outdoor Sample Collection

[0015] According to the pre-planned route, samples were collected outdoors to obtain the latitude and longitude information of the samples.

[0016] S2: Image Preprocessing

[0017] Download the Sentinel-2 remote sensing image of the study area from the ESA website and perform preprocessing such as radiometric calibration, atmospheric correction, and cropping on the downloaded image.

[0018] S3: Sample point selection

[0019] Based on latitude and longitude information obtained from outdoor sampling and prior knowledge, regions of interest (ROIs) are marked using ENVI software, and their coordinates in the image are exported. 60% of the samples are used for model training, 20% for validation, and 20% for testing.

[0020] S4: Training the network

[0021] A network model integrating Transformer and CNN was established, trained using generated samples, tested on a test set, and used to make predictions for other unlabeled samples in the study area.

[0022] Preferably, the power grid building planning land classification method consists of a CNN and a Transformer sequence, and includes the following steps:

[0023] S1: Design a network framework that combines fully connected layers (FC) with CNN. The 1D pixel spectral sequence obtained by the fully connected layer is reshaped into a 3D spectral feature matrix for use as input to the CNN. The FC improves the dimension of the features.

[0024] S2: Extract features from the three levels of CNN and combine them with spectral information after linear transformation to enhance the expressive power of information. At the same time, use it as input to Transformer Encoder to improve the features of CNN by utilizing the global modeling capability of Transformer.

[0025] S3: Finally, skip connections are made between adjacent encoders to enhance the fusion of information at different levels, and pixel classification results are obtained through MLPHead.

[0026] Preferably, the network framework combining FC and CNN consists of FC and CNN. The input spectral information (1×1×4) undergoes a linear transformation through FC, reshaping the features (1×1×256) obtained by the linear transformation into 3D features (4, 4, 64). These 3D features are used as input to the 2D-CNN. The 2D-CNN has a depth of 2. The first convolutional layer contains 64 convolutional kernels, each 1×1 in size, used to increase the dimensionality of the input samples. The second convolutional layer contains 64 convolutional kernels, each 3×3 in size. After the second convolutional layer, an SE module is connected to focus on more effective feature information. The first and second convolutional layers are connected to form a residual unit through residual connections, fusing features from different levels. After the residual unit, a 2×2 average pooling layer is connected to reduce information redundancy and expand the receptive field. The features after two residual units and average pooling are flattened into vector form. Features are extracted from each residual unit, and global average pooling is used to convert the extracted features (n×n×m and n / 2×n / 2×m) into sequence data (1×1×64).

[0027] Preferably, the input to the Transformer model is the original band information. When adding positional encoding, the features extracted by the CNN are combined with the positional encoding and pixel information to further enhance the expressive power of the features using the Transformer. To enable more effective information propagation to deeper layers, skip links are used between adjacent encoders of the Transformer to fully utilize the features between adjacent layers and reduce information loss. The features of the previous encoder and the next encoder are concatenated dimensionally, and then fused using a 1×2 convolution.

[0028] Compared with the prior art, the beneficial effects of the present invention are:

[0029] (1) The model proposed in this invention, which combines Transformer and CNN, has an overall accuracy and Kappa value that are 1.22% and 0.0144 higher than that of Transformer.

[0030] (2) This invention can not only classify two-dimensional convolutional neural networks (2D-CNN), but also classify remote sensing images at the pixel level. Attached Figure Description

[0031] Figure 1 This is a flowchart illustrating the implementation of the method of the present invention;

[0032] Figure 2 It is a 2D-CNN structure;

[0033] Figure 3 For Transformer multi-head attention graph;

[0034] Figure 4 This is a comparison chart of the classification results of the algorithm of this invention and other algorithms on the dataset of the study area. Detailed Implementation

[0035] Example 1

[0036] Please see Figure 1-4 The present invention provides a technical solution:

[0037] A method for classifying power grid building land use by integrating attention and multi-level CNN, comprising the following steps:

[0038] Step 1: Send a multispectral pixel sequence to the FC, use the linear transformation function of the fully connected layer to adjust the dimension of the 1D-spectral sequence, and reshape the transformed 1D-spectral sequence into a 3D-spectral feature matrix;

[0039] Step 2: Extract features from three different levels of the 2D-CNN as part of the Transformer input;

[0040] Step 3: While adding positional encoding, combine the features extracted by the CNN with the positional encoding and pixel information;

[0041] Step 4: Achieve feature fusion by skipping links between adjacent Transformer Encoders;

[0042] Step 5: Finally, the MLP Head obtains the classification results, achieving pixel-level classification;

[0043] Step 6: Classify the study area using different methods, including the following steps.

[0044] S1: Outdoor Sample Collection

[0045] According to the pre-planned route, samples were collected outdoors to obtain the latitude and longitude information of the samples.

[0046] S2: Image Preprocessing

[0047] Download the Sentinel-2 remote sensing image of the study area from the ESA website and perform preprocessing such as radiometric calibration, atmospheric correction, and cropping on the downloaded image.

[0048] S3: Sample point selection

[0049] Based on latitude and longitude information obtained from outdoor sampling and prior knowledge, regions of interest (ROIs) are marked using ENVI software, and their coordinates in the image are exported. 60% of the samples are used for model training, 20% for validation, and 20% for testing.

[0050] S4: Training the network

[0051] A network model integrating Transformer and CNN was established, trained using generated samples, tested on a test set, and used to make predictions for other unlabeled samples in the study area.

[0052] A method for classifying land use in power grid construction planning consists of a CNN and a Transformer architecture. First, a network framework combining a fully connected layer (FC) with a CNN is designed. The 1D pixel spectral sequence obtained by the FC is reconstructed into a 3D spectral feature matrix for input to the CNN. The FC enhances the feature dimensionality, increasing its expressive power and addressing the limitation of 2D-CNNs in pixel-level classification. Second, features from the three layers of the CNN are extracted and combined with the linearly transformed spectral information to enhance information expressiveness. This information is then used as input to the Transformer Encoder, leveraging the Transformer's powerful global modeling capabilities to improve the CNN features. Finally, skip connections are established between adjacent encoder layers to enhance the fusion of information from different levels. Pixel classification results are obtained through an MLP Head.

[0053] Another network framework combining FC and CNN consists of FC and CNN. The input spectral information (1×1×4) undergoes a linear transformation through FC, reshaping the resulting features (1×1×256) into 3D features (4, 4, 64). These 3D features are used as input to the 2D-CNN. The 2D-CNN has a depth of 2. The first convolutional layer contains 64 kernels, each 1×1 in size, used to increase the dimensionality of the input samples. The second convolutional layer also contains 64 kernels, each 3×3 in size. After the second convolutional layer, a SE module is connected to focus on more effective feature information. Residual connections connect the first and second convolutional layers to form residual units, fusing features from different levels. After the residual units, a 2×2 average pooling layer is applied to reduce information redundancy and expand the receptive field. The features after two residual units and average pooling are flattened into vector form. Features are extracted from each residual unit, and global average pooling is used to convert the extracted features (n×n×m and n / 2×n / 2×m) into sequence data (1×1×64).

[0054] The Transformer model takes raw band information as input. When adding positional encoding, it combines the features extracted by the CNN with the positional encoding and pixel information, further enhancing the expressive power of the features using the Transformer. To enable more effective information propagation to deeper layers, skip links are used between adjacent encoders in the Transformer, fully utilizing the features between adjacent layers and reducing information loss. The features from the previous and next encoders are concatenated dimensionally, and then fused using a 1×2 convolution.

Claims

1. A method for classifying power grid building land use by integrating attention and multi-level CNN, characterized in that: Includes the following steps: Step 1: Send a multispectral pixel sequence to the FC, use the linear transformation function of the fully connected layer to adjust the dimension of the 1D-spectral sequence, and reshape the transformed 1D-spectral sequence into a 3D-spectral feature matrix; Step 2: Extract features from three different levels of the 2D-CNN as part of the Transformer input; Step 3: While adding positional encoding, combine the features extracted by the CNN with the positional encoding and pixel information; Step 4: Achieve feature fusion by skipping links between adjacent Transformer Encoders; Step 5: Finally, the MLP Head obtains the classification results, achieving pixel-level classification; Step Six: Classify the study area using different methods; Step six involves classifying the study area using different methods, including the following steps: S1: Outdoor Sample Collection According to the pre-planned route, go outdoors to collect samples and obtain the latitude and longitude information of the samples; S2: Image Preprocessing Download the Sentinel-2 remote sensing image of the study area from the ESA website, and perform radiometric calibration, atmospheric correction, and cropping preprocessing on the downloaded image; S3: Sample point selection Based on the latitude and longitude information obtained from outdoor sampling and prior knowledge, the region of interest is marked using ENVI software and the coordinates of the region of interest in the image are exported. 60% of the samples are used to train the model, 20% of the samples are used for validation, and 20% are used for testing. S4: Training the network A network model integrating Transformer and CNN was established, trained using generated samples, tested on a test set, and made predictions on other unlabeled samples in the study area. The power grid construction land classification method consists of a CNN and a Transformer, and includes the following steps: SS1: Design a network framework that combines fully connected layers (FC) with CNN. The 1D pixel spectral sequence obtained by the fully connected layer is reshaped into a 3D spectral feature matrix for use as input to the CNN. The FC improves the dimension of the features. SS2: Extracts features from the three levels of CNN and combines them with spectral information after linear transformation to enhance the expressive power of information. At the same time, it serves as input to the Transformer Encoder, utilizing the global modeling capabilities of the Transformer to improve the features of the CNN. SS3: Finally, skip connections are made between adjacent encoders to enhance the fusion of information at different levels, and pixel classification results are obtained through the MLP Head; The network framework combining FC and CNN consists of FC and CNN. The input spectral information (1×1×4) undergoes a linear transformation through the FC, reshaping the resulting features (1×1×256) into 3D features (4, 4, 64). These 3D features are used as input to the 2D-CNN, which has a depth of 2. The first convolutional layer contains 64 kernels, each 1×1 in size, and is used to increase the dimensionality of the input samples. The second convolutional layer contains 64 kernels, each 3×3 in size. The second convolutional layer is followed by an SE module to focus on more effective feature information. The first and second convolutional layers are combined into a residual unit through residual connections to fuse features from different levels. After the residual unit, a 2×2 average pooling layer is connected to reduce information redundancy and expand the receptive field. The features after the two residual units and average pooling are flattened into vector form. The features after each residual unit are extracted, and the extracted features (n×n×m and n / 2×n / 2×m) are converted into sequence data (1×1×64) by global average pooling. The input to the Transformer model is the original band information. While adding position encoding, the features extracted by CNN are combined with the position encoding and pixel information. The Transformer is used to further improve the expressive power of the features. In order to make information propagate to deeper layers more effectively, skip links are used between adjacent encoders of the Transformer to make full use of the features between adjacent layers and reduce information loss. The features of the previous encoder and the next encoder are concatenated in dimension, and then the features of the two encoders are fused by a 1×2 convolution.

Images