Knowledge migration type high-resolution remote sensing image building change detection method and system
By constructing a multi-layer attention network MLA-Net and a dynamic domain loss function, combined with weight pooling, the robustness and transferability issues of building change detection in existing technologies are solved, enabling efficient detection of high-resolution remote sensing images in complex scenes and improving detection accuracy and model generalization ability.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-11
- Publication Date
- 2026-03-10
AI Technical Summary
Existing building change detection technologies have shortcomings in robustness, transferability, and sample adaptability. In particular, the detection accuracy decreases in non-standard datasets and complex scenarios, and the imbalanced sample problem leads to weak model generalization ability.
A knowledge-transfer-based high-resolution remote sensing image building change detection method is adopted. A multi-layer attention network MLA-Net is constructed, which combines a dynamic domain loss function and a weight pool. Through adaptive knowledge accumulation and attention mechanism, the model can be transferred and applied across different datasets, thus solving the sample imbalance problem.
The model's detection accuracy and robustness were improved in various imaging modes and complex scenes, achieving efficient building change detection, reducing error propagation, and enhancing the model's generalization ability.
Smart Images
Figure CN121640263A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The application belongs to the technical field of remote sensing application, and in particular, relates to a knowledge transfer type high-resolution remote sensing image building change detection method and system. BACKGROUND
[0002] Change detection is a basic technology in the field of remote sensing. By comparing remote sensing images of the same area at different times, the position and state changes of the target are analyzed. Buildings, as a core component of land use and urban landscape, change detection is of great significance for disaster assessment (such as identifying building damage after an earthquake), urban expansion planning, and ecological environment monitoring.
[0003] With the development of remote sensing technology, image resolution has been improved from traditional low resolution to high resolution and even super high resolution. Satellite sensors have entered the era of multi-view and multi-modal imaging, providing more micro-scale data for building change detection. However, existing building change detection techniques have the following key problems: (1) Lack of robustness: Traditional methods (such as image difference and change vector analysis) and most deep learning-based methods only verify performance on standard datasets (such as nadir imaging, strict registration correction, and simple scenes), and cannot adapt to non-nadir imaging and complex scenes (such as industrial area occlusion, residential area dense buildings) and images that are not strictly preprocessed; (2) Poor transferability: Existing models rely on specific dataset training and lack general adaptability, i.e., when migrating from one dataset to another, the detection accuracy drops significantly, and there is no method to achieve stable migration application between different datasets; (3) Sample and training defects: Training samples generally have an imbalance problem (fewer change area samples and more background samples), and existing loss functions cannot effectively focus on difficult samples (such as small area building changes and fuzzy edge regions), resulting in weak model generalization ability.
[0004] Therefore, there is an urgent need for a building change detection method that has high robustness, transferability, and sample adaptation ability to meet the processing needs of multi-scene and multi-view high-resolution remote sensing images in reality. SUMMARY
[0005] To solve the above technical problems, the present application provides a knowledge transfer type high-resolution remote sensing image building change detection method and system.
[0006] In a first aspect, the present application provides a knowledge transfer type high-resolution remote sensing image building change detection method, comprising: The adaptive knowledge-driven building change detection model containing multi-layer attention is constructed; the building change detection model comprises a multi-layer attention network and a dynamic domain loss function; the multi-layer attention network is an MLA-Net network; the MLA-Net network adopts a linear backbone structure and an input-output separated encoding branch and decoding branch, and the encoding branch and the decoding branch are integrated with a hollow convolution unit, a batch normalization unit and a ReLU unit; the MLA-Net network comprises a channel attention unit, a spatial attention unit and a weight pool unit; the channel attention unit is used for feature modeling of remote sensing image features at a channel level; the spatial attention unit is used for feature modeling of remote sensing image features at a spatial level; and the weight pool is used for storing solidified model parameters of the MLA-Net network obtained through training of different data sets. A dual-time high-resolution remote sensing image is acquired, including a pre-change image, a post-change image and a binary reference image of the same size as the images, and a dual-time high-resolution remote sensing image data set is constructed; The building change detection model is trained by using the dual-time high-resolution remote sensing image data set; the dual-time images are spliced and then input into the MLA-Net network according to a preset batch size, the MLA-Net network parameters are updated through back propagation based on a dynamic domain loss function calculation criterion, until a preset precision or a preset training period is reached, solidified model parameters are obtained, and the solidified model parameters corresponding to each dual-time high-resolution remote sensing image data set are stored in the weight pool; The remote sensing image to be detected is cropped to obtain an image tile of a target pixel, the matching solidified model parameters are called from the weight pool according to the data set scene of the remote sensing image to be detected and loaded into the MLA-Net network, the image tile is input into the MLA-Net network, a binary change map is output, and the building change detection is completed.
[0007] In a second aspect, the present application provides a knowledge transfer type high-resolution remote sensing image building change detection system, comprising a model construction unit, a data set construction unit, a model training unit and a parameter calling and output unit. The model construction unit is configured to construct an adaptive knowledge-driven building change detection model containing multi-layer attention; the building change detection model comprises a multi-layer attention network and a dynamic domain loss function; the multi-layer attention network is an MLA-Net network; the MLA-Net network adopts a linear backbone structure and an input-output separated encoding branch and decoding branch, and the encoding branch and the decoding branch are integrated with a hollow convolution unit, a batch normalization unit and a ReLU unit; the MLA-Net network comprises a channel attention unit, a spatial attention unit and a weight pool unit; the channel attention unit is configured to perform feature modeling on remote sensing image features at a channel level; the spatial attention unit is configured to perform feature modeling on remote sensing image features at a spatial level; and the weight pool is configured to store solidified model parameters of the MLA-Net network obtained through training of different data sets. The data set construction unit is configured to obtain double-time high-resolution remote sensing images, including pre-change images, post-change images and binary reference images of the same size as the images, and construct a double-time high-resolution remote sensing image data set. The model training unit is configured to train the building change detection model using the double-time high-resolution remote sensing image data set; the double-time images are spliced and input into the MLA-Net network according to a preset batch size; the MLA-Net network parameters are updated through back propagation based on a dynamic domain loss function calculation criterion until a preset accuracy or a preset training period is reached, so as to obtain solidified model parameters; and the solidified model parameters corresponding to each double-time high-resolution remote sensing image data set are stored in the weight pool. The parameter calling and output unit is configured to crop the remote sensing image to be detected to obtain an image tile of a target pixel, call the matching solidified model parameters from the weight pool according to the data set scene of the remote sensing image to be detected, load the solidified model parameters into the MLA-Net network, input the image tile into the MLA-Net network, and output a binary change map to complete building change detection.
[0008] On the basis of the above technical solution, the application can be further improved as follows.
[0009] Further, the channel attention unit comprises: performing global average pooling and global maximum pooling on the input feature map along the spatial dimension to aggregate global information of each channel; outputting channel weights of each channel through a multilayer perceptron for channel weight learning; performing pixel-by-pixel multiplication of the channel weights and the original feature map; and taking the channel-weighted feature map as the input of the spatial attention unit.
[0010] Further, the channel weight of each channel is output by learning the channel weight through the multilayer perceptron, including: splicing the average value of all pixels in each channel obtained by global average pooling and the maximum value of all pixels in each channel obtained by global maximum pooling, inputting the first layer of the multilayer perceptron, and outputting through the ReLU activation function; inputting the reduced features into the second layer of the multilayer perceptron, and outputting two channel weight vectors; adding the two channel weight vectors, and obtaining the final channel weight matrix through the Sigmoid activation function.
[0011] Further, the spatial attention unit includes: performing global average pooling and global maximum pooling on the input feature map along the channel dimension to aggregate the global channel information of each spatial position, integrating the results of the global average pooling and the global maximum pooling, extracting the spatial context relationship through convolution operation, obtaining the spatial weight map through the activation function, performing spatial weighting on the spatial position and pixel by pixel multiplication between the spatial weight and the channel weighted feature map, inputting into the decoding branch of the MLA-Net network, and outputting the binary change map through the operations of the dilated convolution, the batch normalization and the ReLU activation.
[0012] Further, the average value of all channels at each spatial position obtained by global average pooling and the maximum value of all channels at each spatial position obtained by global maximum pooling are spliced along the channel dimension, a convolution kernel smaller than a set size is used to perform convolution operation on the spliced result, and the convolution operation result is mapped to the range of [0, 1] through the Sigmoid activation function to obtain the spatial weight map.
[0013] Further, the dual-time high-resolution remote sensing image dataset includes LEVIR-CD dataset, LEVIR-CD+ dataset, WHU Building Dataset dataset and S2 Looking dataset.
[0014] Further, the dual-time high-resolution remote sensing image dataset is used to train the building change detection model, including data initialization, data initialization, iterative training and weight pool construction; Data initialization: the pre-change image, the post-change image and the binary reference image of the same size as the image of all datasets are uniformly cropped to 256x256 pixels, and are divided into a training set and a validation set in the ratio of 8:2; Data initialization: the initial weights of the MLA-Net network are initialized by He normal distribution, the batch size and the learning rate are set, and the Adam optimizer is used as the optimizer; Iterative training: the DDLF function is used as the loss function for iterative training, and the loss value of the predicted result and the reference image is calculated every iteration, and the model parameters are updated through back propagation; the accuracy is evaluated on the validation set every several rounds, and the model parameters with the optimal accuracy are saved to the weight pool. Weight pool construction: store each group of optimal parameters trained by all dual-phase high-resolution remote sensing image data sets to the weight pool, and mark the corresponding data set scene of the parameters.
[0015] Further, the accuracy of the building change detection model is evaluated on the validation set according to F1, IoU, Kappa and OA.
[0016] Further, the process of updating the MLA-Net network parameters through back propagation includes: calculating the loss value of the prediction result of the MLA-Net network output and the binary reference image, solving the gradient based on the loss value, adjusting the weight and bias parameters of the MLA-Net network according to the gradient, and iterating until the model converges.
[0017] The beneficial effects of the present application are: the compatible design of channel attention and spatial attention enables the model to fully mine deep features and adapt to building change detection tasks of multiple imaging modes and complex scenes; the present application proposes a knowledge accumulation method for attention adaptive modeling using a weight pool, stores the model parameters trained by different data to form a weight pool, and reuses it in new building change detection tasks to realize migration application; in addition, the present application proposes a new loss function, i.e. dynamic domain loss function, which successfully solves the sample imbalance problem; the network fully and efficiently models the image features at the channel and spatial levels, and is adaptive to different data based on the weight pool, which can guide the MLA-Net network to focus on information mining of difficult samples to obtain the optimal training effect; the channel attention, spatial attention and weight pool jointly constitute a new multi-layer attention network, i.e. MLA-Net network, which adopts a linear backbone structure, and the input and output are separated, and this end-to-end strategy can significantly reduce error propagation. BRIEF DESCRIPTION OF DRAWINGS
[0018] Figure 1 The principle diagram of the knowledge transfer type high-resolution remote sensing image building change detection method provided for embodiment 1 of the present application; Figure 2 The overall workflow diagram of the building change detection model; Figure 3 The simulation diagram of the loss value of each model on the LEVIR-CD data set; Figure 4 The simulation diagram of the loss value of each model on the LEVIR-CD+ data set; Figure 5 The simulation diagram of the loss value of each model on the WHU Building data set; Figure 6 The simulation diagram of the loss value of each model on the S2Looking data set; Figure 7 simulation plots of precision of iterations of each model on the LEVIR-CD dataset; Figure 8 simulation plots of precision of iterations of each model on the LEVIR-CD+ dataset; Figure 9 simulation plots of precision of iterations of each model on the WHU Building dataset; Figure 10 simulation plots of precision of iterations of each model on the S2Looking dataset; Figure 11 A principle diagram of the knowledge transfer type high-resolution remote sensing image building change detection system provided by Embodiment 2 of the present application. DETAILED DESCRIPTION
[0019] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. The components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations.
[0020] Embodiment 1 As an embodiment, as shown in the accompanying drawings, to solve the above technical problems, the present embodiment provides a knowledge transfer type high-resolution remote sensing image building change detection method, which comprises the following steps: Figure 1 constructing a self-adaptive knowledge driven building change detection model containing multi-layer attention; the building change detection model comprises a multi-layer attention network and a dynamic domain loss function; the multi-layer attention network is an MLA-Net network; the MLA-Net network adopts a linear backbone structure and an input-output separated encoding branch and decoding branch, and the encoding branch and the decoding branch are integrated with a hollow convolution unit, a batch normalization unit and a ReLU unit; the MLA-Net network comprises a channel attention unit, a spatial attention unit and a weight pool unit; the channel attention unit is used for feature modeling of remote sensing image features at a channel level; the spatial attention unit is used for feature modeling of remote sensing image features at a spatial level; and the weight pool is used for storing solidified model parameters of the MLA-Net network obtained through training of different datasets; obtaining double-time high-resolution remote sensing images, including pre-change images, post-change images and binary reference images of the same size as the images, and constructing a double-time high-resolution remote sensing image dataset; The building change detection model is trained by using a double-time high-resolution remote sensing image dataset; the double-time images are spliced and input into the MLA-Net network according to a preset batch size, the MLA-Net network parameters are updated through back propagation based on a dynamic domain loss function calculation criterion until a preset precision or a preset training period is reached, the solidified model parameters are obtained, and the solidified model parameters corresponding to each double-time high-resolution remote sensing image dataset are stored in a weight pool; The remote sensing image to be detected is cropped to obtain an image tile of a target pixel, the matched solidified model parameters are called from the weight pool according to the data set scene of the remote sensing image to be detected, are loaded into the MLA-Net network, the image tile is input into the MLA-Net network, and a binary change map is output, so that the building change detection is completed.
[0021] The compatible design of the channel attention and the spatial attention enables the model to fully mine deep features and adapt to building change detection tasks of various imaging modes and complex scenes. Figure 2 As shown in a whole workflow diagram of the building change detection model, the application proposes a knowledge accumulation method of attention self-adaptive modeling by using a weight pool, model parameters obtained by training different data are stored to form the weight pool, the initial weights A, B, C and D obtain corresponding target weights 、 、 and , and are reused in a new building change detection task to realize migration application; in addition, the application proposes a new loss function, i.e., a dynamic domain loss function (DDLF), which successfully solves the sample imbalance problem and promotes the model to pay more attention to difficult samples. The channel attention, the spatial attention and the weight pool jointly constitute a new multi-layer attention network, i.e., the MLA-Net network, which fully and efficiently models the image features at the channel and spatial levels, and is self-adaptive to different data based on the weight pool, and the DDLF function can guide the MLA-Net to focus on information mining of difficult samples to obtain the optimal training effect.
[0022] The building change detection task is more sensitive to the spectral, texture and structural features of specific artificial objects, so the application uses new mechanisms and characteristics at the macro and micro levels to analyze, simulate and construct a brain-like perception and cognition model. The neural network should not only reflect the weight adjustment in the current sense, but more importantly, its structure should have variability, plasticity, learning and dynamics, so the application designs an MLA-Net network. The MLA-Net network is a lightweight network with simple structure and few parameters. Since the multiplicative effect in back propagation may cause unstable weight updates when the network is deepened, resulting in poor performance in the change detection task, a linear backbone structure is adopted, and the input and output are separated. This end-to-end strategy can significantly reduce error propagation. Dilation convolution, batch normalization and ReLU operations are used in the encoding and decoding branches to minimize the loss of internal data structure and hierarchical spatial information. The double-phase image pair is input into the network after splicing, and the output end produces a binary change map.
[0023] Optionally, the channel attention unit comprises: performing global average pooling and global maximum pooling on the input feature map along the spatial dimension to aggregate global information of each channel, learning channel weights of each channel through a multilayer perceptron to output the channel weights, performing pixel-by-pixel multiplication of the channel weights and the original feature map channel by channel, and taking the channel-weighted feature map as the input of the spatial attention unit.
[0024] Optionally, the channel weight learning through the multilayer perceptron to output the channel weights of each channel comprises: concatenating the average value of all pixels in each channel obtained by global average pooling and the maximum value of all pixels in each channel obtained by global maximum pooling, inputting the first layer of the multilayer perceptron, and outputting through a ReLU activation function; inputting the reduced features into the second layer of the multilayer perceptron to output two channel weight vectors; adding the two channel weight vectors and passing them through a Sigmoid activation function to obtain the final channel weight matrix.
[0025] Optionally, the spatial attention unit comprises: performing global average pooling and global maximum pooling on the input feature map along the channel dimension to aggregate global channel information of each spatial position, integrating the results of global average pooling and global maximum pooling, extracting spatial context relationships through convolution operation, obtaining a spatial weight map through an activation function, performing spatial weighting on the spatial weight map and the channel-weighted feature map through pixel-by-pixel multiplication of each spatial position, inputting the spatial-weighted feature map into the decoding branch of the MLA-Net network, and outputting a binary change map through dilation convolution, batch normalization and ReLU activation operation.
[0026] Optionally, the average value of all channels at each spatial location obtained by global average pooling and the maximum value of all channels at each spatial location obtained by global max pooling are concatenated along the channel dimension. A convolution operation is performed on the concatenated result using a convolution kernel smaller than a set size. The convolution operation result is then mapped to the range [0,1] using the Sigmoid activation function to obtain the spatial weight map.
[0027] Optional, dual-temporal high-resolution remote sensing image datasets include the LEVIR-CD dataset, the LEVIR-CD+ dataset, the WHUBuildingDataset dataset, and the S2Looking dataset.
[0028] The dataset contains training image data of 256*256 pixels before and after the changes, as well as a binary reference image of the same size. (LEVIR-CD dataset) Optionally, the building change detection model can be trained using a dual-temporal high-resolution remote sensing image dataset, including data initialization, data initialization, iterative training and weight pool construction; Data initialization: All images before and after the change, as well as binary reference images of the same size as the images, in the dataset are uniformly cropped to 256×256 pixels and divided into training set and validation set in an 8:2 ratio; Data initialization: The initial weights of the MLA-Net network are initialized using the He normal distribution, and the batch size and learning rate are set. The Adam optimizer is used. Iterative training: Using the DDLF function as the loss function, iterative training is performed. In each iteration, the loss value between the prediction result and the reference image is calculated, and the model parameters are updated through backpropagation. Every few rounds, the accuracy is evaluated on the validation set, and the model parameters with the best accuracy are saved to the weight pool. Weight pool construction: Store the optimal parameters of each group obtained from training all dual-temporal high-resolution remote sensing image datasets into the weight pool, and label the dataset scene corresponding to the parameters.
[0029] The changed and unchanged images and the reference image form a model input, which is input into the MLA-Net network according to the set size of batch training (for example, the size is 8, that is, 8 groups of such inputs), and in the training process, the MLA-Net network obtains a model prediction result through encoding, attention feature enhancement and decoding processes in the network. The prediction result and the reference image are calculated for a prediction deviation in a loss function (evaluated by a loss value), and a back propagation gradient is obtained to perform a reverse parameter update on the MLA-Net network. The loss value of the prediction deviation is detected to evaluate the model training process, and when a target accuracy is reached or a specified training period is completed, a solidified model is obtained. The model trained on each data set obtains different solidified model parameters, which are stored in a weight pool. When image testing is performed, the weight parameters can be selected according to the source of the image. The large-scale images before and after the change are cropped to a tile size of 256*256 to be input into the model to obtain a prediction result.
[0030] Optionally, the accuracy of the building change detection model is evaluated according to F1, IoU, Kappa and OA on the validation set.
[0031] Optionally, the process of updating the MLA-Net network parameters through back propagation includes: calculating the loss value of the prediction result output by the MLA-Net network and the binary reference image, solving the gradient based on the loss value, adjusting the weight and bias parameters of the MLA-Net network according to the gradient, and iterating until the model converges.
[0032] Seven models were trained using the LEVIR-CD, LEVIR-CD+, WHU Building and S2Looking three benchmark data sets, and the accuracy and loss value in the training process were recorded. In the training of all models, the DDLF function was introduced, Figure 3 a simulation diagram of the loss value of each model on the LEVIR-CD data set, Figure 4 a simulation diagram of the loss value of each model on the LEVIR-CD+ data set, Figure 5 a simulation diagram of the loss value of each model on the WHU Building data set, Figure 6 a simulation diagram of the loss value of each model on the S2Looking data set, Figure 7 a simulation diagram of the accuracy of each model on the LEVIR-CD data set, Figure 8 a simulation diagram of the accuracy of each model on the LEVIR-CD+ data set, Figure 9 a simulation diagram of the accuracy of each model on the WHU Building data set, Figure 10The simulation chart of the accuracy of each model iteration on the S2Looking dataset, L1 represents the result corresponding to the FCN8s model, L2 represents the result corresponding to the UNet model, L3 represents the result corresponding to the SegNet model, L4 represents the result corresponding to the UNet++ model, L5 represents the result corresponding to the Attention U-Net model, L6 represents the result corresponding to the DeepLabv3+ model, and L7 represents the result corresponding to the MLA-Net model. Figure 6 The result corresponding to the SegNet model is low in accuracy and is not involved in the simulation.
[0033] The results show that the convergence speed of the MLA-Net network method is faster and tends to be stable within about 20 rounds, which may be attributed to the lightweight structure of the network and the efficient attention mechanism. The convergence process of other models is slower and has larger fluctuations, but they can all converge within 50 rounds. A reasonable explanation for this phenomenon is the existence of DDLF. During training, loss and accuracy are the most intuitive and important indicators reflecting the performance of the model. Therefore, it can be preliminarily considered that the internal design principle of the MLA-Net network is reasonable and robust, and DDLF is crucial to the model training process. The quantitative analysis table on the LEVIR-CD dataset is shown in Table 1, the quantitative analysis table on the LEVIR-CD+ dataset is shown in Table 2, the quantitative analysis table on the S2Looking dataset is shown in Table 3, and the quantitative analysis table on the WHU Building dataset is shown in Table 4.
[0034] Table 1 Quantitative analysis table on the LEVIR-CD dataset
[0035] Table 2 Quantitative analysis table on the LEVIR-CD+ dataset
[0036] Table 3 Quantitative analysis table on the S2Looking dataset
[0037] Table 4 Quantitative analysis table on the WHU Building dataset
[0038] The quantitative evaluation indicators of the prediction results of all models on the test set are calculated, and the results are reported in Tables 1 and 2. At the same time, several examples are selected for visualization, and the receiver operating characteristic curve is drawn to intuitively evaluate the model performance. As shown in the accompanying drawings, Fig. 1 is a diagram of the receiver operating characteristic curve of the MLA-Net model on the LEVIR-CD dataset, Fig. 2 is a diagram of the receiver operating characteristic curve of the MLA-Net model on the LEVIR-CD+ dataset, Fig. 3 is a diagram of the receiver operating characteristic curve of the MLA-Net model on the S2Looking dataset, and Fig. 4 is a diagram of the receiver operating characteristic curve of the MLA-Net model on the WHU Building dataset. Figure 4As can be seen from the statistical results, MLA-Net leads in almost all evaluation indicators. On the four datasets, the average values of F1, IoU, Kappa and OA are 0.818, 0.726, 0.784 and 0.953, respectively. The above four indicators fully demonstrate the excellent performance of the MLA-Net network in this case, where the model is trained for only 50 rounds and the number of patches in the test set is sufficient. In addition, the F1 value performs best, indicating that the MLA-Net network achieves a good balance between detection accuracy and target extraction integrity. It is noted that the MA and FA of the MLA-Net network are both low, indicating that it can accurately locate the change area under the joint attention mechanism. Some models without attention perform poorly in MA and FA. Visualization results show that the MLA-Net detection result has clear and accurate edges and high building integrity, as shown in FIG. 6. Figure 4 The box part. It is worth noting that MLA-Net is very robust to interference factors in images, and there are fewer false detection points in the detection results. The ROC curve reflects the discrimination ability of the model, and the AUC value of MLA-Net is the largest, showing that it has the best overall performance and the strongest generalization ability for the prediction probability of each pixel.
[0039] Optionally, all spatial attention mechanisms are performed before the splicing operation, which can effectively fuse low-level and high-level features, facilitate cross-layer context information transmission, obtain compact global information, and guide the network to focus on the area related to the building change detection task. Convolutional operations process images layer by layer and transmit information, use the information collected by the receptive field to form high-dimensional output, and then obtain feature maps through nonlinear activation.
[0040] Embodiment 2 Based on the same principle as the knowledge transfer type high-resolution remote sensing image building change detection method shown in Embodiment 1 of the present application, as shown in FIG. 6, Figure 11 As shown in FIG. 6, the present application also provides a knowledge transfer type high-resolution remote sensing image building change detection system in the embodiment, which comprises a model construction unit, a data set construction unit, a model training unit and a parameter calling and output unit. The model construction unit is configured to construct an adaptive knowledge-driven building change detection model comprising a multi-layer attention mechanism; the building change detection model comprises a multi-layer attention network and a dynamic domain loss function; the multi-layer attention network is an MLA-Net network; the MLA-Net network adopts a linear backbone structure and an input-output separated encoding branch and decoding branch, and the encoding branch and the decoding branch both integrate a dilated convolution unit, a batch normalization unit and a ReLU unit; the MLA-Net network comprises a channel attention unit, a spatial attention unit and a weight pool unit; the channel attention unit is configured to model features of remote sensing images at a channel level; the spatial attention unit is configured to model features of remote sensing images at a spatial level; and the weight pool is configured to store solidified model parameters of the MLA-Net network obtained through training of different data sets. The data set construction unit is configured to obtain double-time high-resolution remote sensing images, including pre-change images, post-change images and binary reference images of the same size as the images, and construct a double-time high-resolution remote sensing image data set. The model training unit is configured to train the building change detection model using the double-time high-resolution remote sensing image data set; the double-time images are spliced and input into the MLA-Net network according to a preset batch size; the MLA-Net network parameters are updated through back propagation based on a dynamic domain loss function calculation criterion until a preset accuracy or a preset training period is reached, and solidified model parameters are obtained; the solidified model parameters corresponding to each double-time high-resolution remote sensing image data set are stored in the weight pool. The parameter calling and output unit is configured to crop the remote sensing image to be detected to obtain an image tile of a target pixel, call matching solidified model parameters from the weight pool according to a data set scene of the remote sensing image to be detected, load the solidified model parameters into the MLA-Net network, input the image tile into the MLA-Net network, and output a binary change map to complete building change detection.
[0041] Optionally, the channel attention unit comprises: performing global average pooling and global maximum pooling on an input feature map along a spatial dimension to aggregate global information of each channel; outputting channel weights of each channel through a multilayer perceptron; performing pixel-by-pixel multiplication of the channel weights and an original feature map; and taking the channel-weighted feature map as an input of the spatial attention unit.
[0042] Optionally, the channel weight of each channel is output by learning the channel weight through a multilayer perceptron, comprising: concatenating the average value of all pixels in each channel obtained by global average pooling and the maximum value of all pixels in each channel obtained by global maximum pooling, inputting the first layer of the multilayer perceptron, and outputting through a ReLU activation function; inputting the reduced features into the second layer of the multilayer perceptron, and outputting two channel weight vectors; adding the two channel weight vectors, and then inputting the result into a Sigmoid activation function to obtain the final channel weight matrix.
[0043] Optionally, the spatial attention unit comprises: performing global average pooling and global maximum pooling on the input feature map along the channel dimension to aggregate the global channel information of each spatial position, integrating the results of the global average pooling and the global maximum pooling, extracting the spatial context relationship through convolution operation, obtaining the spatial weight map through the activation function, performing spatial weighting on the spatial position and pixel by pixel multiplication between the spatial weight and the channel weighted feature map, inputting into the decoding branch of the MLA-Net network, and outputting the binary change map through the operations of atrous convolution, batch normalization and ReLU activation.
[0044] Optionally, the average value of all channels at each spatial position obtained by global average pooling and the maximum value of all channels at each spatial position obtained by global maximum pooling are concatenated along the channel dimension, a convolution kernel smaller than a set size is used to perform convolution operation on the concatenated result, and the convolution operation result is mapped to the range [0, 1] through a Sigmoid activation function to obtain a spatial weight map.
[0045] Optionally, the dual-time high-resolution remote sensing image dataset includes LEVIR-CD dataset, LEVIR-CD+ dataset, WHU Building Dataset dataset and S2 Looking dataset.
[0046] Optionally, the building change detection model is trained using the dual-time high-resolution remote sensing image dataset, comprising data initialization, data initialization, iterative training and weight pool construction. Data initialization: the pre-change image, the post-change image and the binary reference image of the same size as the image of all datasets are uniformly cropped to 256x256 pixels, and divided into a training set and a validation set in a ratio of 8:2; Data initialization: the initial weights of the MLA-Net network are initialized with He normal distribution, the batch size and the learning rate are set, and the Adam optimizer is used as the optimizer; Iterative training: the DDLF function is used as the loss function for iterative training, and the loss value of the predicted result and the reference image is calculated every iteration, and the model parameters are updated through back propagation; the accuracy is evaluated on the validation set every several rounds, and the model parameters with the optimal accuracy are saved to the weight pool; Weight pool construction: store each group of optimal parameters obtained by training all dual-phase high-resolution remote sensing image data sets to the weight pool, and label the corresponding data set scene of the parameters.
[0047] Optionally, the accuracy of the building change detection model is evaluated on the validation set according to F1, IoU, Kappa and OA.
[0048] Optionally, the process of updating the MLA-Net network parameters through back propagation includes: calculating the loss value of the prediction result of the MLA-Net network output and the binary reference image, solving the gradient based on the loss value, adjusting the weight and bias parameters of the MLA-Net network according to the gradient, and iterating until the model converges.
[0049] The above is only a preferred embodiment of the present application and is not used to limit the present application. For those skilled in the art, the present application can have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.
Claims
1. A method for detecting building changes in high-resolution remote sensing images of the knowledge transfer type, characterized in that, The application relates to a building change detection model based on adaptive knowledge and multi-layer attention. The building change detection model comprises a multi-layer attention network and a dynamic domain loss function. The multi-layer attention network is an MLA-Net network. The MLA-Net network adopts a linear backbone structure and an input-output separated encoding branch and decoding branch, and the encoding branch and the decoding branch are integrated with a hollow convolution unit, a batch normalization unit and a ReLU unit. The MLA-Net network comprises a channel attention unit, a spatial attention unit and a weight pool unit. The channel attention unit is used for feature modeling of remote sensing image features at a channel level. The spatial attention unit is used for feature modeling of remote sensing image features at a spatial level. The weight pool is used for storing solidified model parameters of the MLA-Net network obtained through training of different data sets. Double-time high-resolution remote sensing images, including pre-change images, post-change images and binary reference images of the same size as the images, are obtained to construct a double-time high-resolution remote sensing image data set.
2. The knowledge transfer type high-resolution remote sensing image building change detection method according to claim 1, characterized in that, The building change detection model is trained by using the double-time high-resolution remote sensing image data set.
3. The knowledge transfer-based high-resolution remote sensing image building change detection method according to claim 2, characterized in that, The double-time images are spliced and then input into the MLA-Net network according to a preset batch size. The MLA-Net network parameters are updated through back propagation based on a dynamic domain loss function calculation criterion until a preset accuracy or a preset training period is reached. The solidified model parameters corresponding to each double-time high-resolution remote sensing image data set are stored in the weight pool. The MLA-Net network is loaded with the matched solidified model parameters called from the weight pool according to the data set scene of the remote sensing image to be detected. The image tile of the target pixel is obtained by cropping the remote sensing image to be detected. The binary change map is output by inputting the image tile into the MLA-Net network, and the building change detection is completed. The channel attention unit comprises: performing global average pooling and global maximum pooling on the input feature map along the spatial dimension to aggregate the global information of each channel. The channel weight of each channel is output through a multilayer perceptron. The channel weight and the original feature map are multiplied pixel by pixel, and the channel-weighted feature map is taken as the input of the spatial attention unit. The channel weight of each channel is output through a multilayer perceptron. The average value of all pixels in each channel obtained through global average pooling and the maximum value of all pixels in each channel obtained through global maximum pooling are spliced and sent to the first layer of the multilayer perceptron. The output is activated by a ReLU function. The reduced features are sent to the second layer of the multilayer perceptron. Two channel weight vectors are output. The two channel weight vectors are added and then activated by a Sigmoid function to obtain the final channel weight matrix.
4. The knowledge transfer type high-resolution remote sensing image building change detection method according to claim 1, characterized in that, The spatial attention unit comprises: performing global average pooling and global maximum pooling on the input feature map along the channel dimension to aggregate global channel information of each spatial position, integrating results of the global average pooling and the global maximum pooling, extracting spatial context relationships through convolution operation, obtaining a spatial weight map through an activation function, performing spatial weighting on the spatial weight and the channel weighted feature map through pixel-by-pixel multiplication of each spatial position, inputting the spatial weight into a decoding branch of the MLA-Net network, and outputting a binary change map through a dilated convolution, batch normalization and ReLU activation operation.
5. The knowledge transfer-based high-resolution remote sensing image building change detection method according to claim 4, characterized in that, The average value of all channels of each spatial position obtained through the global average pooling and the maximum value of all channels of each spatial position obtained through the global maximum pooling are spliced along the channel dimension, a convolution operation is performed on the spliced result using a convolution kernel smaller than a set size, and a spatial weight map is obtained by mapping the convolution operation result to the range [0, 1] through a Sigmoid activation function.
6. The knowledge transfer-based high-resolution remote sensing image building change detection method according to claim 1, wherein, The dual-temporal high-resolution remote sensing image dataset includes a LEVIR-CD dataset, a LEVIR-CD+ dataset, a WHU Building Dataset dataset and an S2 Looking dataset.
7. The knowledge transfer-based high-resolution remote sensing image building change detection method according to claim 1, wherein, The dual-temporal high-resolution remote sensing image dataset is used to train the building change detection model, including data initialization, data initialization, iterative training and weight pool construction. Data initialization: the pre-change image, the post-change image and the binary reference image of the same size as the image of all datasets are uniformly cropped to 256x256 pixels, and are divided into a training set and a validation set in a ratio of 8:2; Data initialization: the initial weights of the MLA-Net network are initialized using He normal distribution, the batch size and the learning rate are set, and the Adam optimizer is used as the optimizer; Iterative training: the DDLF function is used as the loss function for iterative training, and the loss value of the predicted result and the reference image is calculated every iteration, and the model parameters are updated through back propagation; the accuracy is evaluated on the validation set every several rounds, and the model parameters with the optimal accuracy are saved to the weight pool; Weight pool construction: the optimal parameters of each group obtained by training all dual-temporal high-resolution remote sensing image datasets are stored in the weight pool, and the corresponding dataset scene of the parameters is labeled.
8. The knowledge transfer-based high-resolution remote sensing image building change detection method according to claim 7, wherein, The accuracy of the building change detection model is evaluated according to F1, IoU, Kappa and OA on the validation set.
9. The knowledge transfer-based high-resolution remote sensing image building change detection method according to claim 1, wherein, The process of updating the MLA-Net network parameters through back propagation comprises: calculating the loss value of the predicted result output by the MLA-Net network and the binary reference image, solving the gradient based on the loss value, adjusting the weight and bias parameters of the MLA-Net network according to the gradient, and iterating until the model converges.
10. A knowledge transfer type high resolution remote sensing image building change detection system, characterized in that, The building change detection model comprises a model construction unit, a dataset construction unit, a model training unit, and a parameter calling and output unit. The model construction unit is configured to construct an adaptive knowledge-driven building change detection model containing multi-layer attention; the building change detection model comprises a multi-layer attention network and a dynamic domain loss function; the multi-layer attention network is an MLA-Net network; the MLA-Net network adopts a linear backbone structure and an input-output separated encoding branch and decoding branch, and the encoding branch and the decoding branch are integrated with a hollow convolution unit, a batch normalization unit and a ReLU unit; the MLA-Net network comprises a channel attention unit, a spatial attention unit and a weight pool unit; the channel attention unit is configured to perform feature modeling on remote sensing image features at a channel level; the spatial attention unit is configured to perform feature modeling on remote sensing image features at a spatial level; the weight pool is configured to store solidified model parameters of the MLA-Net network obtained through training of different data sets; The data set construction unit is configured to obtain double-time high-resolution remote sensing images, including pre-change images, post-change images and binary reference images of the same size as the images, and construct a double-time high-resolution remote sensing image data set; The model training unit is configured to train the building change detection model using the double-time high-resolution remote sensing image data set; the double-time images are spliced and input into the MLA-Net network according to a preset batch size, the MLA-Net network parameters are updated through back propagation based on a dynamic domain loss function calculation criterion, until a preset accuracy or a preset training period is reached, the solidified model parameters are obtained, the solidified model parameters corresponding to each double-time high-resolution remote sensing image data set are stored in the weight pool; The parameter calling and output unit is configured to crop the remote sensing image to be detected to obtain an image tile of a target pixel, call the matching solidified model parameters from the weight pool according to the data set scene of the remote sensing image to be detected, load the solidified model parameters into the MLA-Net network, input the image tile into the MLA-Net network, and output a binary change map to complete building change detection.