A remote sensing image building change detection method based on feature enhancement network

Abstract

The application discloses a kind of based on feature enhancement network's remote sensing image building change detection method, comprising the following steps: first step: preparation dataset;Second step: data enhancement is carried out;Third step: network model is built and training, fourth step: building change detection.The application introduces visual transformer structure, space and channel attention, u-shaped residual module and enhanced feature extraction module and self-attention feature fusion module to realize the full fusion of different building information, which can better distinguish buildings of different regular shapes and sizes to prevent false detection and missed detection, while improving the feature extraction capability of buildings of different shapes and their edge details.Compared with different advanced algorithms, such as BIT, Changeformer, etc., the application has higher F1 score and Kappa coefficient.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of remote sensing image processing, and particularly relates to a remote sensing image building change detection method based on a feature enhancement network. BACKGROUND

[0002] At present, since entering the 21st century, people's living standards have gradually improved, and the development of urbanization is becoming more and more important. As one of the symbols of urban construction, the reconstruction, demolition, growth, and reduction of buildings can largely represent the changes of the city, and have important significance for the planning and management of the city. Change detection refers to the process of observing the state difference of the same geographical location at different times, and the change detection of buildings also has important significance for land resource utilization, post-disaster reconstruction, etc.

[0003] The change detection of buildings can be generally divided into traditional change detection algorithms and deep learning-based change detection algorithms. In the traditional method, the change detection algorithm of remote sensing images can be roughly divided into direct comparison method and classification comparison method. The direct comparison method mainly analyzes the geometric features, spectral texture, etc. of the building, and obtains the change information from the image by direct comparison. The classification comparison algorithm first classifies the remote sensing images of different periods, and then compares to determine the final changed and unchanged areas. The accuracy of change detection is mainly determined by the classification result.

[0004] However, as buildings become more and more complex, and remote sensing images contain more and more object information, traditional methods are increasingly difficult to meet the needs. The traditional building change detection method is mostly dependent on manually constructed features and is easily disturbed by different information such as noise, image registration, etc. Moreover, the features constructed by the traditional method can only fit relatively simple buildings, and it is difficult to fit complex and abstract building features. In addition, the traditional algorithm needs to rely on a large amount of professional knowledge and experience in different fields by professional personnel, which consumes a lot of manpower and material resources, and the current mostly based on field investigation makes the efficiency very low. Therefore, automatic, intelligent, and rapid building change detection methods are increasingly needed.

[0005] With the development of space remote sensing technology, deep learning has been applied to change detection. Deep learning has strong modeling and learning ability, and can extract features and perform end-to-end change detection on images through a series of models (such as UNet, STANet, etc.), improving detection accuracy and speed.

[0006] However, some existing model algorithms have insufficient feature extraction capability, resulting in problems such as loss of edge detail changes of buildings, omission of small-scale target buildings in complex backgrounds, poor detection effect of irregularly shaped target buildings, and difficulty in distinguishing changes between different buildings with similar positions when detecting changes in buildings. SUMMARY

[0007] The purpose of the present application is to provide a remote sensing image building change detection method based on a feature enhancement network, which can improve the feature representation capability of the network to further improve the change detection accuracy.

[0008] The technical scheme adopted by the present application is:

[0009] A remote sensing image building change detection method based on a feature enhancement network, specifically comprising the following steps: first, preparing a data set, collecting a public change detection data set CDD, the data set containing a validation set, a training set and a test set. Each subset contains three folders A, B and OUT, corresponding to the pre-change image, the post-change image and the real changed building label respectively, and each image is 256x256 pixels in size.

[0010] As a further improvement of the technical solution: second, data augmentation. In order to enhance the network's ability to recognize buildings in different scenes and the network's robustness, and increase the network's generalization ability, methods such as horizontal flipping and rotation are used to augment the data.

[0011] As a further improvement of the technical solution: third, building a network model and training. The input image is first input into the feature extractor to extract building features. The feature extractor consists of three parts: a primary feature extractor, an enhanced feature extractor and a ResNet decoder.

[0012] As a further improvement of the technical solution: the primary feature extractor mainly consists of Unet encoding and visual transformer structure. Each encoding block of Unet contains two convolutional layers, and each convolutional layer outputs a feature map. The two feature maps are input into VTS to obtain a larger receptive field and enhance the feature representation capability, and finally five feature maps are output.

[0013] As a further improvement of the technical solution: the fifth output feature map is input into the enhanced feature extractor to further enhance the feature representation capability. The enhanced feature extractor consists of four modules, namely the spatial and channel attention module, the U-shaped residual module, the enhanced feature extraction module and the self-attention feature fusion module, which are combined to further enhance the network's ability to represent building features and robustness.

[0014] As a further improvement of the technical solution: the space and channel attention module is composed of space attention and channel attention, which increases the attention to the feature map in the channel dimension and the spatial dimension, and can effectively enhance the expression ability of the network to important architectural features.

[0015] As a further improvement of the technical solution: the U-shaped residual module can better capture global and local information to enhance architectural feature extraction, and the enhanced feature extractor can improve the ability to extract representative architectural features from the channel and spatial dimensions.

[0016] As a further improvement of the technical solution: the self-attention feature fusion module fully merges feature information through different operations such as "sum", "difference" and "splicing".

[0017] As a further improvement of the technical solution: the output of the enhanced feature extractor and the primary feature extractor is input into the ResNet decoder for decoding, and finally two feature maps are output.

[0018] As a further improvement of the technical solution: the two output feature maps of the feature extractor are input into the cross-channel context semantic aggregation module for sufficient fusion of channel information.

[0019] As a further improvement of the technical solution: the output feature map of the cross-channel context semantic aggregation module is input into the convolutional layer to obtain the final change detection map.

[0020] As a further improvement of the technical solution: the loss function used in network training is a combination of cross-entropy loss, Dice loss and Focal loss, to improve the influence of the imbalance between changing buildings and unchanged buildings.

[0021] As a further improvement of the technical solution: the test set samples are input into the trained network model to predict the building change map.

[0022] The present application provides spatial correlation for buildings with different level feature maps by introducing a visual transformer structure, and enhances the recognition ability of the network for buildings in different positions; further, by introducing space and channel attention, irrelevant background information is filtered from the spatial dimension and the channel dimension, and the detection ability for small scale buildings is improved.

[0023] Further, by designing a u-shaped residual module and an enhanced feature extraction module, the feature extraction ability for buildings of different shapes and edge details is improved. A self-attention feature fusion module is proposed to fully fuse different architectural information, which can better distinguish buildings of different regular shapes and sizes to prevent false detection and missed detection.

[0024] Further, a cross-channel context semantic aggregation module is designed to perform information aggregation in the channel dimension, which can better utilize context semantic information, and in order to reduce information loss when merging feature maps and improve the network's building detection capability. Compared with different advanced algorithms such as BIT and Changeformer, the present application has higher F1 score and Kappa coefficient. BRIEF DESCRIPTION OF DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0026] Figure 1 Variation detection flowchart in the specific embodiment of the present application;

[0027] Figure 2 Network model structure diagram in the specific embodiment of the present application;

[0028] Figure 3 Visual transformer structure diagram in the specific embodiment of the present application;

[0029] Figure 4 Spatial and channel attention module structure diagram in the specific embodiment of the present application;

[0030] Figure 5 U-shaped residual module and enhanced feature extraction module structure diagram in the specific embodiment of the present application;

[0031] Figure 6 Self-attention feature fusion module structure diagram in the specific embodiment of the present application;

[0032] Figure 7 Cross-channel context semantic aggregation module structure diagram in the specific embodiment of the present application;

[0033] Figure 8 Building change detection result comparison diagram of the present application and other existing advanced methods. DETAILED DESCRIPTION

[0034] With reference to the drawings and the embodiments of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort belong to the scope of protection of the present application.

[0035] As shown in FIGS. Figure 1 , 2 and 3, the present application comprises the following steps:

[0036] Step 1: Prepare the data set; Step 1, prepare the data set. Collect the public change detection data set CDD, which contains 11 seasonal change remote sensing image pairs. The size of 7 image pairs is 4725x2700 pixels, and the size of 4 image pairs is 1900x1000 pixels. The resolution is 0.03m / pixel~1m / pixel. The image size is cropped to 256x256 pixels, of which 10000 are used for training, 3000 are used for verification, and 3000 are used for testing.

[0037] Step 2: Data augmentation; in order to enhance the network's ability to recognize buildings in different scenes and the robustness of the network, and increase the generalization ability of the network, methods such as horizontal flip and rotation are used to perform data augmentation on the images.

[0038] Step 3: Build the network model and train; the model structure of the feature enhancement network is shown in FIG. Figure 2 The input image is first input into the feature extractor to extract building features. The feature extractor is composed of three parts: a primary feature extractor, an enhanced feature extractor, and a ResNet decoder.

[0039] The primary feature extractor mainly includes Unet coding and visual transformer structure. Each coding block of Unet contains two convolutional layers, and each convolutional layer outputs a feature map. The two feature maps are input into VTS to obtain a larger receptive field and enhance the ability of feature representation. Finally, the primary feature extractor outputs five feature maps. The fifth feature is input into the enhanced feature extractor for further feature enhancement. The enhanced feature extractor is composed of four modules. The first four output features and the fifth enhanced feature are input into the ResNet structure for decoding. After decoding, the feature extractor can output two feature maps of the same size. The two feature maps are input into the cross-channel context semantic aggregation module for fusion. Finally, we obtain the output change detection map by two convolutional layers with a convolution kernel size of 3x3 and a convolutional layer with a convolution kernel size of 1x1. The channel number of the output change map is 2.

[0040] Primary feature extractor

[0041] like Figure 2 As shown in (b), the primary feature extractor employs a UNet encoding structure and a Visual Transformer (VTS) structure. Each encoding block contains two convolutional layers, and each convolutional layer outputs a feature map. These two feature maps are then input into the VTS. The structure of the VTS is as follows: Figure 3 As shown. Because the receptive field of the feature map after convolution is larger than that before convolution, the second feature map contains richer semantic information. After patch embedding, the first feature is used as the query vector. The second feature is used as the key vector and value vector. Multi-head attention is set to 12 in this invention. At the same time, the first feature is used as the position matrix, which is superimposed with the second feature as input. In addition, to reduce network parameters, the transformer block is set to 1. After scaling the feature map, the size is 768×16×16. Then, they are fed into the transposed convolutional layer to change the size and number of channels of the feature map. The final output feature map has the same size as the input feature map. The ResNet feature decoder mainly uses a part of the ResNet18 network structure. First, the size of the input feature map is expanded to twice the original size by transposed convolution with a kernel size of 7×7. Then the feature map is fed into the residual module, and a Dropout layer is added after the residual module to reduce overfitting. Then it is concatenated with the feature map of the corresponding scale size and then fed into the residual module. This process is repeated until the final output feature map is obtained.

[0042] Spatial and channel attention modules

[0043] like Figure 4 As shown, channel attention uses adaptive average pooling to pool each channel of the input feature map. Two fully connected layers are then used to reduce feature parameters, and a ReLU function is used to add non-linearity. The result of the fully connected layers is fed into a sigmoid function to perform weight normalization, and then the weights are multiplied by each element of the input feature map to obtain the channel attention feature map. Spatial attention uses average pooling and max pooling. Max pooling is used to extract representative features of buildings, while average pooling considers global feature information. Different sized pooling kernels are used to perform the pooling operation. The left branch uses a 3×3 pooling kernel, and the middle branch uses a 5×5 pooling kernel. Smaller pooling kernels capture finer target features, while larger kernels extract richer target features. After performing the pooling operation, a 1×1 convolutional layer is used to adjust the number of channels. Then, the outputs of the different pooling operations are concatenated to fuse information. The result is fed into two convolutional layers to obtain initial weights, and then the final weights are calculated using a sigmoid function. Finally, the spatial attention and channel attention are added together to obtain the final output.

[0044] U-shaped residual module and enhanced feature extraction module

[0045] As shown in Figure 5 (a), the U-shaped residual module is divided into two parts: the upper branch and the lower branch. The size of the input feature map is reduced to half of its original size by using a max-pooling layer with a pooling kernel size of 2x2. Then deep semantic features are extracted by four convolutional layers. According to the skip connection, the information above is used to reduce the information loss. The outputs of the fourth and third convolutional layers are concatenated and input into a convolutional layer. After performing three identical operations, the output is up-sampled. Then the output and the input feature are added. For the lower branch, the operations are the same except that the max-pooling is replaced by average-pooling.

[0046] As shown in Figure 5 (b), the enhanced feature extraction module is divided into the left branch and the right branch. The left branch mainly extracts the feature information in the spatial dimension, and the right branch mainly excavates the feature in the channel dimension. The left branch first compresses the input feature channel to 1 by 1x1 convolution. Then its spatial features are extracted by two convolutional layers. The result is delivered to the sigmoid function to perform weight normalization. Multiply the weight and the input feature to assign a weight to each feature element. Finally, the left branch output is obtained by addition. The left side of the right branch uses the average pooling operation to pool each channel information of the input feature. The right side of the right branch uses the max-pooling operation, which is to consider the feature information from the global and local angles. Then two 1x1 convolutional layers are used to compress and expand the channel, and the result is activated by the sigmoid function. After multiplying the activation result and the input feature, the left feature and the right feature are added to obtain the final output.

[0047] Self-attention feature fusion module

[0048] As shown in Figure 6 To better fuse feature information, three operations are used: addition operation, subtraction operation and concatenation. Then, two convolutional layers are used to obtain deep representative features. After adjusting the number of channels by 1x1 convolution, according to the self-attention, it is known that this mechanism can make pixels in different positions correlate with each other, and can well identify buildings. Therefore, the subtraction branch is regarded as the query vector, the addition branch is regarded as the key vector, and the concatenation branch is regarded as the value vector. The reshape and transpose operations are performed on the subtraction output, and the reshape operation is performed on the addition output. Then, they are multiplied, and the multiplication result is activated by the sigmoid function, and then two convolutional layers are used to fuse the feature information of the concatenation branch. Then it is multiplied by the maximum weight matrix to construct the spatial long-distance dependency relationship, and then the multiplication result is reshaped. Finally, the result and the two input features are added element by element, and the final output is regarded as the weighted sum of the input feature and all position features.

[0049] Cross-channel context semantic aggregation module

[0050] As Figure 7 shown in the module, the intermediate feature map is obtained by channel splicing the left feature map and the right feature map, then the size of the intermediate feature is compressed to 1x1 through an adaptive average pooling layer, and the channel number is adjusted through a 1x1 convolution layer, then the convolution output is spliced with it, the splicing result is fed into a 1x1 convolution layer, and then the convolution result is input into the right branch to perform connection and fusion, and a sigmoid function is used to normalize the weight matrix, and the channel information can be fully aggregated and fused through the left and right branches. Two convolution layers are used to extract multi-scale features of the left branch features and the right branch features. Finally, the weight matrix and the convolution result are multiplied, and then element-wise addition is performed, so that the channel information of the output feature map can be well converged.

[0051] When training the network, the loss function used is a combination of cross-entropy loss, Dice loss and Focal loss, as follows, to improve the influence of the imbalance between the changing buildings and the unchanged buildings.

[0052]

[0053]

[0054]

[0055] L=L bce +δL dc +φL fc (4)

[0056] y n Where represents the real ground surface change, p n represents the predicted building change, H, W respectively represent the height and width of the image. Alpha is a hyperparameter, alpha >= 0, p is the estimated probability of the model, and the value range is [0, 1]. Delta, phi are used to balance the loss L bce , L dc , L fc .

[0057] Fourth step, building change detection. After the network training is completed and converges, the test set sample images are used to output the change detection map.

[0058] In order to verify the effectiveness of the present application, the training and testing of different algorithm models are carried out using the CDD data set, and the training and testing are carried out in the same environment. The algorithms used for comparison are STANET, SNUNET, BIT, Changeformer and IDET. The test is carried out through five evaluation indexes, and the evaluation indexes are overall accuracy, precision, recall, F1 score and Kappa coefficient. F1 is the harmonic mean of precision and recall, and the larger the value is, the better. The specific evaluation index results are shown in Table 1.

[0059]

[0060] As can be seen from Table 1, the method of the present application is higher than the five existing advanced methods in all indexes, which proves the effectiveness of the method of the present application.

[0061] The building change detection result comparison diagram of the method of the present application and other existing methods is shown in Figure 8 .

[0062] As can be seen from Figure 8 , in the change detection result of the method of the present application, the proposed model has more complete and accurate detection results. And for irregularly shaped target buildings and changes between buildings with similar positions, it can be well distinguished. The method can also filter the influence of background noise and enhance the detection ability of small scale target buildings.

[0063] The present application has been described in detail above with reference to the drawings and examples, but the present application is not limited to the above examples. Within the scope of knowledge possessed by those skilled in the art, various changes can be made using the technical content disclosed above without departing from the purpose of the present application. The contents not described in detail in the present application can adopt the prior art.

[0064] In the description of the present application, it should be noted that for orientation words such as the terms "center", "transverse", "longitudinal", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise" and the like indicate the orientation and positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and cannot be understood as limiting the specific protection scope of the present application.

[0065] It has to be noted that, as used herein, the terms "comprising", "including", "containing", "characterized by", "comprised of", "comprising" or grammatical equivalents are used herein to mean that the process, method, object or apparatus of the application contains, but is not limited to, the stated features, integers, steps or components. Thus, a process, method, object or apparatus that comprises a list of elements is not necessarily limited to only those elements on the list.

[0066] It is to be understood that the above description and the examples are intended to be illustrative and not restrictive. Many other embodiments will be apparent to those of skill in the art upon reviewing the above description. The scope of the application should, therefore, be determined not with reference to the above description, but instead should be determined with reference to the appended claims, along with their full scope of equivalents.

Claims

1. A method for detecting building changes in remote sensing images based on feature enhancement networks, characterized in that: Includes the following steps: Step 1: Prepare the dataset: Collect the publicly available change detection dataset CDD. The dataset contains a validation set, a training set, and a test set. Each subset contains three folders: A, B, and OUT, which correspond to the images before the change, the images after the change, and the labels of the buildings that actually changed, respectively. Each image is 256×256 pixels in size. Step 2: Data augmentation: In order to enhance the network's ability to recognize buildings in different scenarios and the network's robustness, and to increase the network's generalization ability, horizontal flipping and rotation methods are used to augment the images. Step 3: Build and train the network model; the network model includes: Feature Extractor: The input image is first fed into a feature extractor to extract architectural features; the feature extractor includes: Primary Feature Extractor: Used to enhance feature representation capabilities; the primary feature extractor mainly consists of Unet encoding and a visual transformer structure; each encoding block of Unet contains two convolutional layers, each convolutional layer outputs a feature map; these two feature maps are input into the VTS to obtain a larger receptive field and enhance the feature representation capability, ultimately outputting five feature maps; specifically, it includes the following steps: After patch embedding, the first feature is used as the query vector; Use the second feature as both the key vector and the value vector; With multi-head attention set to 12, the first feature is used as the position matrix, which is then superimposed with the second feature and used as input. The transformer block is set to 1. After scaling the feature maps, their size is 768×16×16. Then, they are fed into a transposed convolutional layer to change the size and number of channels of the feature maps. The final output feature map has the same size as the input feature map. Enhanced Feature Extractor: Further enhances feature representation capabilities; The ResNet decoder takes the outputs of the enhanced feature extractor and the primary feature extractor as inputs to the ResNet decoder for decoding, ultimately outputting two feature maps. The cross-channel contextual semantic aggregation module is used to fully integrate channel information. In this module, the intermediate feature map is obtained by concatenating the left and right features. Then, an adaptive average pooling layer compresses the size of the intermediate feature to 1×1, and a 1×1 convolutional layer adjusts its channel count. The convolutional output is then concatenated with the intermediate feature map, and the concatenated result is fed into a 1×1 convolutional layer. The convolutional result is then input into the right branch for connection and fusion. A sigmoid function is used to normalize the weight matrix. Channel information can be fully aggregated and fused through the left and right branches. Two convolutional layers are used to extract multi-scale features from the left and right branches. Finally, the weight matrix and the convolutional result are multiplied and added element-wise to ensure good convergence of channel information in the output feature map. Step 4: Building change detection; Input the test set samples into the trained network model to predict building change maps.

2. The method for detecting building changes in remote sensing images based on feature enhancement networks according to claim 1, characterized in that, The fifth output feature map is input into the enhanced feature extractor to further enhance the feature representation capability. The enhanced feature extractor consists of four modules: a spatial and channel attention module, a U-shaped residual module, an enhanced feature extraction module, and a self-attention feature fusion module. Together, they further enhance the network's ability to represent architectural features and its robustness. The spatial and channel attention module consists of spatial attention and channel attention, adding attention to the feature map in both channel and spatial dimensions, which can effectively enhance the network's ability to represent important architectural features. The U-shaped residual module is used to enhance architectural feature extraction and better capture global and local information. The enhanced feature extraction module is used to improve the ability to extract representative architectural features from the channel and spatial dimensions. The self-attention feature fusion module fully merges feature information through summation, subtraction, and concatenation.

3. The method for detecting building changes in remote sensing images based on feature enhancement networks according to claim 2, characterized in that, The ResNet feature decoder primarily uses a portion of the ResNet18 network structure. First, it expands the input feature map to twice its original size using a transposed convolution with a kernel size of 7×7. Then, the feature map is input into the residual module, after which a Dropout layer is added to reduce overfitting. Finally, it is concatenated with a feature map of the corresponding scale and then input back into the residual module. This process is repeated until the final output feature map is obtained.

4. The method for detecting building changes in remote sensing images based on feature enhancement networks according to claim 2, characterized in that, The spatial and channel attention modules are used in practice through the following steps: Channel attention uses an adaptive average pooling operation to pool each channel of the input feature map; Then, two fully connected layers are used to reduce the feature parameters, and the ReLU function is used to increase non-linearity; The result of the fully connected layer is fed into the sigmoid function to perform weight normalization. Then, the weights are multiplied by each element of the input feature map to obtain the channel attention feature map.

5. The method for detecting building changes in remote sensing images based on feature enhancement networks according to claim 2, characterized in that, The loss function used in the network training of step three is a combination of cross-entropy loss, Dice loss, and Focal loss, which is used to mitigate the impact of the imbalance between changing and invariant buildings, as shown in the following formula: L=L bce +δL dc +φL fc (4) y n Where p represents the actual changes on the Earth's surface. n The predicted building changes are represented by H and W, which represent the height and width of the image, respectively; α is a hyperparameter, α≥0, and p is the estimated probability of the model, with a value range of [0,1]. δ and φ are used to balance losses; L bce It is the cross-entropy loss function, L dc L fc These are the Dice loss function and the Focal loss function, respectively.

6. The method for detecting building changes in remote sensing images based on feature enhancement networks according to claim 2, characterized in that: The U-shaped residual module is divided into two parts: an upper branch and a lower branch; its specific use includes the following steps: The size of the input feature map is reduced to half its original size using a max pooling layer with a pooling kernel size of 2×2. Then, deep semantic features are extracted through four convolutional layers; based on skip connections, contextual information is used to reduce information loss; the outputs of the fourth and third convolutional layers are concatenated and input into the convolutional layer. After performing the same operation three times, the output is upsampled; then the output and input features are added together. The following branches operate the same as the branches above, except that average pooling is used instead of max pooling.

7. The method for detecting building changes in remote sensing images based on feature enhancement networks according to claim 2, characterized in that: The enhanced feature extraction module is divided into a left branch and a right branch. The left branch mainly extracts feature information in the spatial dimension, while the right branch mainly mines features in the channel dimension. The left branch first compresses the input feature channels to 1 using a 1×1 convolution. Then, it extracts its spatial features through two convolutional layers. The result is fed into the sigmoid function to perform weight normalization. The weights are multiplied by the input features to assign weights to each feature element. Finally, the left branch output is obtained through addition. The left side of the right branch uses average pooling to pool the information of each channel of the input feature. The right side of the right branch uses max pooling to comprehensively consider feature information from both global and local perspectives. Then, two 1×1 convolutional layers are used to compress and expand the channels, and the result is activated by the sigmoid function. After multiplying the activation result with the input features, the left and right features are added to obtain the final output.

8. The method for detecting building changes in remote sensing images based on feature enhancement networks according to claim 2, characterized in that: By adjusting the number of channels using 1×1 convolutions, and based on self-attention, it is known that this mechanism can correlate pixels at different locations, thus effectively identifying buildings. Therefore, the self-attention feature fusion module works as follows: the subtraction branch is treated as a query vector, the addition branch as a key vector, and the concatenation branch as a value vector; reshape and transpose operations are performed on the subtraction output; then, they are multiplied, and the multiplication result is activated using the sigmoid function; finally, two convolutional layers are used to fuse the feature information of the concatenation branch. Then, it is multiplied by the final weight matrix to construct long-range spatial dependencies, and the multiplication result is reshaped. Finally, the result is added to the two input features element by element, and the final output is regarded as a weighted sum of the input features and all positional features.

Images