A remote sensing image building change detection method, device, medium and equipment

Abstract

The application discloses a remote sensing image building change detection method and device, medium and equipment, the method comprises the following steps: constructing a building change detection model, the building change detection model comprises two inputs, and is used for inputting a remote sensing image respectively; the building change detection model is trained; the original remote sensing image data of two periods is input into the building change detection model, and a detection result is obtained; the building change detection model comprises an encoder module and a decoder module; a stage connection module is further arranged between the encoder module and the decoder module; a feature offset learning module is arranged in the decoding network; the feature offset learning module is used for aligning the features from bottom to top; the extraction and learning ability of the features are fully utilized; and the accuracy of remote sensing image building change detection is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and particularly relates to a remote sensing image building change detection method, device, medium and equipment. BACKGROUND

[0002] Building change detection is an operation of analyzing and determining the building changes in two different periods of images, and the purpose is to distinguish the changed and unchanged pixels in the remote sensing images of the same region at different times. With the continuous improvement of remote sensing technology, the resolution of remote sensing images is continuously improved, and it is possible to obtain more detailed ground building conditions from remote sensing images. The acquisition and analysis of ground building conditions are of great significance for city planning management, post-disaster emergency decision-making and regional development planning. The continuous progress of deep learning technology and the continuous improvement of computer hardware computing power make it possible to detect building changes based on deep learning.

[0003] In the prior art, there is a processing method that first performs corresponding merging, difference or stacking processing on the image, and then inputs the neural network for subsequent processing, but these operations do not consider the feature differences of the source image, which is easy to cause poor expression ability of the features. On the other hand, the fusion or stacking of the image may produce some synthetic information which has some adverse effects on the subsequent feature learning. Another processing method is that the change detection uses a double-input model, but how to fully utilize the features of the remote sensing image to improve the accuracy of the change detection is still one of the research hotspots in the field. SUMMARY

[0004] In view of the problems existing in the prior art, the present application provides a remote sensing image building change detection method, device, medium and equipment, which realizes the splicing of shallow feature maps and deep feature maps through a stage connection module, is conducive to information fusion, realizes better feature splicing through a feature offset learning module, fully utilizes the feature extraction and learning ability, and has better detection performance and higher precision compared with the traditional change detection method. The technical scheme is as follows:

[0005] In the first aspect, a remote sensing image building change detection method is provided, comprising:

[0006] A building change detection model is constructed, and the building change detection model comprises two inputs for inputting one remote sensing image respectively;

[0007] The building change detection model is trained;

[0008] Two period original remote sensing image data are input into the building change detection model to obtain a detection result;

[0009] The building change detection model comprises:

[0010] an encoder module comprising a first encoding network for processing original remote sensing images of a first period and a second encoding network for processing original remote sensing images of a second period, the first encoding network and the second encoding network being a weight-sharing twin structure, the encoding network comprising a plurality of stages in sequence from input to output, the plurality of stages implementing a plurality of times of consecutive down-sampling on the original remote sensing images, and the encoding network outputting an encoding output feature map after a plurality of times of down-sampling as a decoding input feature map of a decoder module;

[0011] a decoder module comprising a decoding network, the decoding network comprising a plurality of stages in sequence from input to output, the plurality of stages implementing a plurality of times of consecutive up-sampling on the decoding input feature map;

[0012] The encoder module and the decoder module are further provided with a stage connection module, and the stage connection module is configured to transmit the feature map after convolution of each stage of the encoding network to the corresponding stage of the decoding network, and fuse the feature map after up-sampling of the corresponding stage of the decoding network.

[0013] The feature map after convolution of each stage of the encoding network is transmitted to the corresponding stage of the decoding network, and fused with the feature map after up-sampling of the corresponding stage of the decoding network, comprising:

[0014] The two feature maps to be fused have the same width and height;

[0015] After the two feature maps to be fused are spliced, the feature offset learning module is used to realize fusion optimization in the decoding network, and the feature offset learning module comprises a preset convolution processing layer, batch normalization and a ReLU activation function connected in sequence, the preset convolution operation of the preset convolution processing layer comprises adding a learnable parameter in convolution calculation to adjust the sampling position of the convolution kernel in space according to different positions, different scales and different deformation targets in the feature map, and the learnable parameter is obtained through training.

[0016] In some embodiments, the preset convolution operation is:

[0017]

[0018] where Δp n is a learnable parameter representing an offset prediction value, P0 represents the coordinates of the center of the convolution kernel relative to the upper left corner of the input feature map, P n is the offset of the convolution kernel element relative to the center of the convolution kernel, w is the value of the corresponding position of the convolution kernel, is the value of the position relative to the center P n of the convolution kernel in the convolution kernel, x is the input feature map, is the P0+P n +Δp in the input feature map relative to the upper left corner of the input feature mapn a value of a position, y is a convolution output result.

[0019] In some embodiments, the stage connection module further comprises an integrated channel module for optimizing the transmission of the encoded output feature map to the features of the decoding network, the integrated channel module comprising:

[0020] a first branch for receiving the encoded output feature map F, splitting the feature map F into n sub-maps; splicing each sub-map to form a new feature map F0, performing a channel attention mechanism on the feature map F0 to obtain an attention coefficient CA1 for the feature map F0;

[0021] a second branch for receiving the encoded output feature map F and summing the feature map F to obtain a global feature map, performing a channel attention mechanism on the global feature map to obtain a global attention coefficient CA2 for the global feature map;

[0022] a fusion module for repeatedly splicing and adding CA2 output by the second branch and CA1 output by the first branch n times to obtain a feature map fusion representation output by the integrated channel module;

[0023] an output module for generating an output feature map through upsampling and a 1x1 convolution layer.

[0024] In some embodiments, the splitting of the feature map F into n sub-maps; splicing each sub-map to form a new feature map F0 comprises:

[0025] obtaining the original channel number b of the feature map F;

[0026] dividing the b channels into n parts;

[0027] splicing the n parts to form a feature map F0 with n channels.

[0028] In some embodiments, the summing of the feature map F to obtain a global feature map comprises:

[0029] first, using the torch.stack function to stack the input feature map list to generate a new tensor;

[0030] then using the torch.sum function to sum the stacked tensor along dimension 0.

[0031] In some embodiments, each stage of the encoding network comprises a 3*3 convolution layer, a RULE function and a 2*2 max pooling layer connected in sequence, and each stage of the decoding network comprises an upsampling layer and a feature offset learning module connected in sequence.

[0032] In some embodiments, the loss function used in the training process of the building change detection model comprises a binary cross-entropy loss function and a Dice loss function, and the binary cross-entropy loss function is:

[0033]

[0034] where n is the number of samples, y i is the true label, P(y i ) is the predicted label value corresponding to the true label y i , and y i takes a value of 0 or 1.

[0035] The Dice loss function is: where TP is the number of correctly detected changed pixels, FP is the number of incorrectly detected changed pixels, and FN is the number of incorrectly detected unchanged pixels.

[0036] In a second aspect, a remote sensing image building change detection device is provided, comprising:

[0037] a model construction unit configured to construct a building change detection model, the building change detection model comprising two inputs for inputting one remote sensing image respectively;

[0038] a model training unit configured to train the building change detection model;

[0039] a building change detection unit configured to input two-period original remote sensing image data into the building change detection model to obtain a detection result;

[0040] The building change detection model comprises:

[0041] an encoder module comprising a first encoding network for processing a first-period original remote sensing image and a second encoding network for processing a second-period original remote sensing image, the first encoding network and the second encoding network being a weight-sharing twin structure, the encoding network comprising a plurality of stages in sequence from input to output, the plurality of stages performing continuous a times of down-sampling on the original remote sensing image, and the encoding output feature map outputted after a times of down-sampling of the encoding network being used as a decoding input feature map of a decoder module;

[0042] a decoder module comprising a decoding network, the decoding network comprising a plurality of stages in sequence from input to output, the plurality of stages performing continuous a times of up-sampling on the decoding input feature map;

[0043] The encoder module and the decoder module are further provided with a stage connection module, the stage connection module being configured to transmit the feature map after convolution of each stage of the encoding network to the corresponding stage of the decoding network, and fuse the feature map after up-sampling of the corresponding stage of the decoding network.

[0044] The feature map after convolution of each stage of the encoding network is fused with the feature map after upsampling of the corresponding stage of the decoding network, including:

[0045] The two feature maps to be fused have the same width and height;

[0046] The two feature maps to be fused are spliced and then fused and optimized in the decoding network through a feature offset learning module, the feature offset learning module including a preset convolution processing layer, batch normalization and a ReLU activation function connected in sequence, the preset convolution operation of the preset convolution processing layer including adding a learnable parameter in convolution calculation for adjusting the sampling position of the convolution kernel in space according to different positions, different scales and different deformation targets in the feature map, the learnable parameter being obtained through training.

[0047] In a third aspect, an electronic device is provided, and the electronic device includes:

[0048] a processor;

[0049] a memory for storing processor-executable instructions;

[0050] The processor implements the remote sensing image building change detection method of the first aspect by running the executable instructions.

[0051] In a fourth aspect, a computer-readable storage medium is provided, and the medium stores computer instructions, and the instructions are executed by a processor to implement the steps of the remote sensing image building change detection method of the first aspect.

[0052] The remote sensing image building change detection method, device, medium and equipment have the following beneficial effects: the shallow feature map and the deep feature map are spliced through the stage connection module, which is conducive to information fusion, reduces the number of network parameters, and alleviates the overfitting phenomenon. The features from the bottom up are aligned through the feature offset learning module, so that the feature splicing is better, the small target is better recognized, and the edge irregularity problem is solved. The features transmitted from the encoding network to the decoding network are optimized through the integrated channel module, thereby improving the accuracy of change detection. BRIEF DESCRIPTION OF DRAWINGS

[0053] Figure 1 is a structural schematic diagram of a building change detection model in the embodiment of the application;

[0054] Figure 2 is a structural schematic diagram of an integrated channel module in the embodiment of the application;

[0055] Figure 3is a loss function curve diagram of a building change detection model training process in an embodiment of the application;

[0056] Figure 4 is a detection result schematic diagram of four groups of typical areas in the building change detection model experimental results in an embodiment of the application. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical scheme and advantages of the application more clear, the application will be further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the application and not to limit the application.

[0058] The remote sensing image building change detection method provided by the application is characterized in that it comprises:

[0059] A building change detection model is constructed, which comprises two inputs for inputting one remote sensing image respectively;

[0060] The building change detection model is trained;

[0061] Two period original remote sensing image data are input into the building change detection model to obtain a detection result;

[0062] Referring to Figure 1 , the building change detection model comprises:

[0063] An encoder module comprises a first encoding network for processing a first period original remote sensing image and a second encoding network for processing a second period original remote sensing image, the first encoding network and the second encoding network are weight-sharing twin structures, the encoding network comprises a stages in sequence from input to output, the stages realize a times of consecutive down-sampling on the original remote sensing image, and the encoding output feature map output after a times of down-sampling of the encoding network is taken as a decoding input feature map of a decoder module;

[0064] The decoder module comprises a decoding network, the decoding network comprises a stages in sequence from input to output, the stages realize a times of consecutive up-sampling on the decoding input feature map, and the decoding network adopts a 1×1 convolution to adjust the output channel number after the a stages, and outputs a building change detection result;

[0065] The stage connection module is used for transmitting the feature map after convolution of each stage of the encoding network to the corresponding stage of the decoding network, and fusing the feature map after up-sampling of the corresponding stage of the decoding network, the corresponding stage satisfies that the width and height of the feature map after up-sampling of the corresponding stage of the decoding network are the same as the width and height of the feature map transmitted to the corresponding stage of the decoding network of the encoding network.

[0066] wherein the feature map after convolution of each stage of the encoding network is fused with the feature map after up-sampling of the corresponding stage of the decoding network, including:

[0067] After the two feature maps to be fused are spliced, fusion optimization is realized in the decoding network through a feature offset learning module, the feature offset learning module including a preset convolution processing layer, batch normalization and a ReLU activation function connected in sequence, the preset convolution operation adopted by the preset convolution processing layer including adding a learnable parameter in convolution calculation for adjusting the sampling position of the convolution kernel in space according to different positions, different scales and different deformation targets in the feature map, the learnable parameter being obtained through training.

[0068] In the embodiment of the application, the building change detection model adopts an encoding-decoding structure, the encoding part has 4 stages of down-sampling, and the decoding part continuously performs 4 stages of up-sampling with the output of the last stage of the encoding part as the input, so as to realize feature extraction and change detection of the original remote sensing image data of two periods. Further, in order to optimize the feature extraction of the decoding part, the feature map after convolution of each stage of the encoding network is fused into the corresponding stage of the decoding network. In the application, the shallow feature map is spliced with the deep feature map through a stage connection module, which is conducive to information fusion, reduces the number of network parameters, and alleviates the overfitting phenomenon.

[0069] It can be understood that the feature map after convolution of each stage of the encoding network is fused with the feature map after up-sampling of the corresponding stage of the decoding network, the two feature maps corresponding to the fusion have the same size, and after the two feature maps to be fused are spliced, fusion optimization is realized through the preset convolution processing layer, batch normalization and the ReLU activation function.

[0070] Further, referring to Figure 1 , each stage of the above-mentioned encoding network includes a 3*3 convolution layer, a RULE function and a 2*2 maximum pooling layer connected in sequence, and each stage of the decoding network includes an up-sampling layer and a feature offset learning module connected in sequence.

[0071] It can be understood that the down-sampling multiple realized by each stage of the above-mentioned encoding network and the up-sampling multiple realized by each stage of the decoding network are consistent, for example, both are 2, the number of channels realized by the convolution layer of each stage of the above-mentioned encoding network is increased, the number of channels realized by each feature offset learning module of the decoding network is reduced, and the change multiple of the number of channels realized by the convolution layer of each stage of the above-mentioned encoding network and the change multiple of the number of channels realized by each feature offset learning module of the decoding network are in a 1 / 2 relationship, for example, the convolution layer output channel number of each stage of the encoding network is changed to 2 times the convolution layer input channel number, and the feature offset learning module input channel number of the decoding network is changed to 4 times the convolution layer output channel number.

[0072] Assuming that a = 4 above, the encoding network is sequentially the first stage, the second stage, the third stage, and the fourth stage, and if the feature map width and height are reduced by half in each stage, then the four stages achieve 16 times down-sampling, and the decoding network is sequentially the first stage, the second stage, the third stage, and the fourth stage, and the output of the fourth stage of the encoding network is transmitted to the input of the first stage of the decoding network; further, the encoding-decoding structure of the building change detection model is described, taking the sampling multiple as 2, the channel number change multiple of the feature offset learning module as 4, and the channel number change multiple of the 3*3 convolution layer of the encoding network as 2, as shown in FIG. 2. Figure 1

[0073] In the encoding network, the input remote sensing image sequentially passes through a 3*3 convolution and a 2*2 pooling in the first stage to double the channel number of the feature map while reducing the width and height by one time, obtaining a first feature map with a width and height of 256*256 and a channel number of 64; after sequentially passing through the second stage, the third stage, and the fourth stage, a fourth feature map with a width and height of 32*32 and a channel number of 512 is obtained;

[0074] The feature map output after 4 times down-sampling of the encoding network is taken as the input feature map of the decoder module after 3*3 convolution;

[0075] In the decoding network, the input feature map sequentially passes through an up-sampling layer and a feature offset learning module in the first stage, wherein after passing through the first stage up-sampling layer, an A1 feature map with a size of 512*64*64 is obtained, and under the action of the stage connection module, the A1 feature map is spliced with the feature map (512*64*64) after convolution of the fourth stage of the encoding network, and after passing through the feature offset learning module of the first stage of the decoding network, a feature map with a width and height of 64*64 and a channel number of 256 output by the first stage of the decoding network is obtained, denoted as A feature map, sequentially, after passing through the up-sampling layer and the feature offset learning module of the second stage of the decoding network, a B feature map with a width and height of 128*128 and a channel number of 128 is obtained, after passing through the up-sampling layer and the feature offset learning module of the third stage, a C feature map with a width and height of 256*256 and a channel number of 64 is obtained, and after passing through the up-sampling layer and the feature offset learning module of the fourth stage, a D feature map with a width and height of 512*512 and a channel number of 32 is obtained;

[0076] After passing through all stages of the decoding network, the output channel number is adjusted by using a 1*1 convolution to output the building change detection result.

[0077] ​wherein the preset convolution processing layer, batch normalization and ReLU activation function constitute a feature migration learning module (FML). The traditional convolution operation is a sliding window operation on a regular square convolution kernel, and the same weight is applied at each position to extract local features of an image. However, in the presence of shape changes, rotation or distortion, the traditional convolution operation may not adapt well to these changes, resulting in inaccuracy of feature extraction. The convolution operation in the above preset convolution processing layer adopted in the embodiments of the present application realizes flexible adjustment of the sampling position of the convolution kernel in space according to the scale or deformation of the target by introducing learnable parameters. In the embodiments of the present application, the features of the feature map after convolution at each stage and the feature map after upsampling at the corresponding stage of the decoding network are aligned up and down by the preset convolution operation, avoiding the leakage of low-level feature information into high-level features when the two feature maps are fused, which causes shape changes or distortions and further leads to the loss of feature information during subsequent upsampling, affecting the extraction of building edges, realizing better feature splicing, and ultimately better recognizing small targets and solving the problem of irregular edges.

[0078] Further, the preset convolution operation in the above feature migration learning module is:

[0079]

[0080] wherein Δp n is a learnable parameter representing a migration prediction value, P0 represents the coordinates of the center of the convolution kernel relative to the upper left corner of the input feature map, P n is the migration of the convolution kernel element relative to the center of the convolution kernel, w is the value of the corresponding position of the convolution kernel, x is the input feature map, and y is the convolution output result.

[0081] In the embodiments of the present application, the implementation process of the traditional convolution is as follows: first, sampling on the input feature mapping x with the convolution kernel, and second, summing the weighted sampling values of w; wherein R is a set of migrations of convolution kernel elements relative to the center of the convolution kernel, for example, R = {(-1, -1), (-1, 0),..., (0, 1), (1, 1)} defines a 3*3 convolution kernel.

[0082] For each position P0 on the output feature mapping y, there is:

[0083] wherein p n enumerates the positions in R;

[0084] The preset convolution operation in the present application is as follows:

[0085]

[0086] wherein Δp nP0 is the offset of the center of the convolution kernel relative to the upper left corner of the input feature map, P n R is the offset of the convolution kernel element relative to the center of the convolution kernel, and for a 3x3 convolution, the parameter R belongs to (-1,-1), (-1.0), …, (1,1), and the upper left corner is (-1,-1) by default, and the lower right corner is (1,1).

[0087] Referring to Figure 2 In an embodiment, the stage connection module further comprises an integrated channel module for optimizing the transmission of the encoded output feature map to the features of the decoding network, and the integrated channel module comprises:

[0088] A first branch for receiving the encoded output feature map F, splitting the feature map F into n subgraphs, splicing each subgraph to form a new feature map F0, and performing a channel attention mechanism on the feature map F0 to obtain an attention coefficient CA1 for the feature map F0;

[0089] A second branch for receiving the encoded output feature map F and summing the feature map F to obtain a global feature map, and performing a channel attention mechanism on the global feature map to obtain a global attention coefficient CA2 for the global feature map;

[0090] A fusion module for repeatedly splicing and adding CA2 output by the second branch and CA1 output by the first branch n times to obtain a feature map fusion representation output by the integrated channel module;

[0091] An output module for generating an output feature map through upsampling and a 1x1 convolution layer.

[0092] In the embodiments of the present application, the stage connection module provides an integrated channel module for transmitting the features of the encoded output feature map to the decoding network in a more optimal strategy. The integrated channel module, the first branch splits and reorganizes the original encoded output feature map F to obtain n feature maps F0, performs a channel attention mechanism on each feature map F0 to obtain an attention coefficient CA1, and the second branch is used to obtain a global attention coefficient CA2. The two attention coefficients cooperate to fully capture features of different scales and improve the representation ability of the features transmitted from the encoding network to the decoding network.

[0093] In an embodiment, in the first branch of the above integrated channel module, the feature map F is split into n subgraphs, and each subgraph is spliced to form a new feature map F0, comprising:

[0094] Obtaining the original channel number b of the feature map F;

[0095] Dividing the b channels into n parts;

[0096] The n parts are spliced ​​together to form a feature map F0 with n channels.

[0097] In one embodiment of this application, the first branch splits and reassembles the feature map F, which can be done by splitting and reassembling the channels of the feature map F. It can be understood that the number of channels of the feature map F is greater than the number of channels of the feature map F0. For example, if the number of channels of the feature map F is 8, it is split into 8 single channels and divided into 4 groups, which are then spliced ​​together to form a 4-channel feature map F0.

[0098] In one implementation, the second branch of the integrated channel module above sums the feature map F to obtain a global feature map, including:

[0099] First, the torch.stack function is used to stack the list of input feature maps to generate a new tensor;

[0100] Then, the torch.sum function is used to sum the stacked tensors along dimension 0.

[0101] In this embodiment, the input feature map list is first stacked using the torch.stack function, generating a new tensor. Then, the torch.sum function is used to sum the results along dimension 0 of the stacked tensor, and the sum is assigned to a variable that is subsequently passed to the channel module.

[0102] In one implementation, the loss functions used during the training of the aforementioned building change detection model include the binary cross-entropy loss function and the Dice loss function. These loss functions combine information from multiple aspects, simultaneously considering both the model's accuracy and the similarity of the segmentation results. The binary cross-entropy loss function compares the difference between the true label and the predicted label. Specifically, for the building change detection task, assuming there are n samples and the true label is y... i Real label y i The corresponding predicted label value is P(y i Its formula can be expressed as follows:

[0103]

[0104] Based on the above formula, it can be observed that the closer the predicted label is to the true label, the smaller the loss function value will be, and vice versa. Building change detection, as a binary classification task, y i The value of can only be 0 or 1, so the loss function only needs to calculate the loss value when the predicted value is 0 and 1 respectively.

[0105] The Dice loss function mainly measures the similarity between the model prediction result and the true label, and its value is the ratio of the intersection size of the predicted value and the true value to the reciprocal of the ratio of their union size. In the building change detection task, its mathematical expression can be expressed as follows:

[0106]

[0107] Where TP is the number of correctly detected changed pixels, FP is the number of false detected changed pixels, and FN is the number of false detected unchanged pixels.

[0108] The following provides experimental data to verify the effectiveness of the remote sensing image building change detection method provided above.

[0109] This paper uses the LEVIR-CD dataset suitable for change detection tasks, which comes from 20 different areas in several cities in Texas, USA. The data has a large time span, so it introduces enough seasonal and light changes, which helps to develop related methods to reduce the impact of irrelevant changes on actual changes. The data consists of 637 pairs of dual-phase image blocks with obvious building growth, which is two orders of magnitude larger than other public datasets in the field, with more than 30,000 independently labeled change instances. In order to facilitate training using GPU, the original image size is divided into 512x512 pixel size, and the cropped image pairs are divided into training set, validation set and test set in the ratio of 7:2:1.

[0110] During the experiment, the hardware environment used is an Intel(R) Core(TM) i7-8700 CPU @ 3.20 GHz processor, 64 GB of memory, and an NVIDIA GeForce RTX 2060 12 GB graphics card; The software environment is: the deep learning framework uses Pytorch, the acceleration library uses CUDA11.2, and the programming language is Python3.7.1.

[0111] When training the building change detection model, the smaller the loss function value represents the better the fitting effect of the model. In order to better find the global minimum loss function value, this paper uses the Adam optimizer and uses the cosine annealing strategy to decay the learning rate. The characteristic of this strategy is that as the independent variable increases, the cosine function will periodically decay the learning rate, and in the later stage of training, a small fixed learning rate will be maintained for more fine-tuned model parameter adjustment. The loss function curve during the training of the building change detection model is shown in Figure 3 As can be seen from the curve, there are several peaks, which first decrease rapidly and then slowly, and in the later stage of training, a small loss value is maintained, which is caused by the use of the cosine annealing strategy.

[0112] To verify the effectiveness of the method proposed in this paper, after the network training is completed, the test set images are input into the network, and the change detection results of each pair of images are output end to end. This paper mainly uses precision, recall and F1 three indicators to measure the effect of the model, precision and recall represent the accuracy and integrity of the model respectively, by taking the harmonic mean of the two model evaluation indicators of precision and recall, a comprehensive evaluation model of the model is obtained. The F1 score ranges from 0 to 1, 0 represents the worst model output effect, and 1 represents the best model output effect.

[0113] The model evaluation indicators are shown in Table 1. In order to verify the influence of the feature offset learning module and the integrated channel module on the overall performance of the network, UNet is used as the basic network, and the feature offset learning module (DUNet) and the integrated channel module (EUNet) are added to the final method for experimental comparison. At the same time, other change detection methods are introduced for related experiments. From the table, it can be seen that the F1 score of the method in this paper is higher than that of other methods, and compared with UNet++, Deeplabv3 and Unet, the F1 is improved by 3.87%, 1.25% and 5.23% respectively. After adding the feature alignment module, the precision decreases by 2.05%, the recall increases by 12.49%, and the F1 increases by 4.14%; after adding the channel attention module, the precision decreases by 1.27%, the recall increases by 11.91%, and the F1 increases by 4.22%. After introducing both, the precision increases by 0.1%, the recall increases by 12.85%, and the F1 increases by 5.23%.

[0114] Table 1 Comparison of experimental results based on LEVIR-CD dataset

[0115]

[0116] The experimental results of LEVIR-CD dataset are selected as Figure 4The four groups of typical regions are shown, the white region is the changed region, the black part is the unchanged region, and the red box represents the region with significant contrast effect. It can be seen from the comparison that the change detection methods with contrast all extract the changed buildings to different degrees. The Deeplabv3 network can effectively detect the changed region, but the generated result shape boundary is irregular, and there is a false detection situation. The classic U-shaped network UNet and UNet++ have improved the boundary of the irregular shape building extraction, but there is still a large error, and the missed detection phenomenon still exists, as shown in the comparison of region 3, the detected changed building has a large difference with the original building shape boundary, and the small building is not successfully detected. The performance of the U-shaped network for irregular shapes is relatively poor, which may be due to the encoder-decoder structure of the network, which loses part of the image details in each down-sampling layer, so in the irregular region shown in region 1, the result of the irregular region is poor. In this paper, irregular convolution is used in the up-sampling part, and channel attention mechanism is introduced in the jump connection part. Through experiments, it can be seen that the detection effect of irregular buildings is obviously improved (region 1), the small target recognition is more accurate and the boundary is smoother (regions 2 and 3), and the building change in the false detection area (region 4) is also correctly presented, so this method can better detect the changed region.

[0117] The embodiment of the present application provides a kind of remote sensing image building change detection device, the device includes:

[0118] Model construction unit, for constructing building change detection model, the building change detection model includes 2 inputs, for respectively input one remote sensing image;

[0119] Model training unit, for training the building change detection model;

[0120] Building change detection unit, for inputting two period original remote sensing image data into the building change detection model, obtains detection result;

[0121] The building change detection model includes:

[0122] Encoder module includes the first encoding network for processing first period original remote sensing image and the second encoding network for processing second period original remote sensing image, the first encoding network and the second encoding network are weight sharing twin structure, the encoding network includes a stage in order from input to output a, a stage realizes continuous a times down-sampling to original remote sensing image, and the encoding output feature map outputted after a times down-sampling of encoding network is used as the decoding input feature map of decoder module;

[0123] The decoder module comprises a decoding network, the decoding network comprises a stages in sequence from input to output, the stages implement a continuous a times of up-sampling on the decoding input feature map, and the decoding network;

[0124] The stage connection module is further arranged between the encoder module and the decoder module, and is used for transmitting the encoding output feature map output by each stage of the encoding network to the corresponding stage of the decoding network and fusing the encoding output feature map with the decoding output feature map of the corresponding stage of the decoding network.

[0125] The fusion of the encoding output feature map output by each stage of the encoding network and the decoding output feature map of the corresponding stage of the decoding network comprises:

[0126] The encoding output feature map and the decoding output feature map to be fused have the same size;

[0127] The encoding output feature map and the decoding output feature map to be fused are spliced and then fused and optimized in the decoding network through a feature offset learning module, the feature offset learning module comprises a preset convolution processing layer, batch normalization and a ReLU activation function connected in sequence, the preset convolution operation of the preset convolution processing layer comprises adding a learnable parameter in convolution calculation to adjust the sampling position of the convolution kernel in space according to different positions, different scales and different deformation targets in the feature map, and the learnable parameter is obtained through training.

[0128] The specific limitations of the remote sensing image building change detection device can be referred to the limitations of the remote sensing image building change detection method in the foregoing, and will not be described herein. It should be noted that: the remote sensing image building change detection device provided in the embodiment is used for building change detection, and only the division of the above functional units is used as an example for illustration, and in actual application, the above functions can be completed by different functional units according to needs, that is, the internal structure of the device is divided into different functional units to complete all or part of the above described functions.

[0129] The embodiment of the application provides an electronic device, which comprises:

[0130] A processor;

[0131] A memory for storing processor executable instructions;

[0132] The processor implements the remote sensing image building change detection method described above by running the executable instructions.

[0133] It can be understood that the processor of the electronic device includes a main processor, such as a CPU, for processing computing data, and in addition, an image processor (GPU) for performing image and graphics related operations on the remote sensing image in the embodiments of the present application. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, an electronic program and a database, and the database is used to store building remote sensing images (including training sample images and remote sensing images to be detected). The internal memory provides an environment for the operating system and computer program in the non-volatile storage medium to run.

[0134] The embodiments of the present application provide a computer readable storage medium, which stores computer instructions, and the instructions are executed by a processor to implement the steps of the remote sensing image building change detection method. For example, the computer readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), a magnetic tape, a floppy disk and an optical data storage node, etc.

[0135] The present application is not limited to the above specific embodiments, and various modifications made by those skilled in the art based on the above concept without creative labor are all within the protection scope of the present application.

Claims

1. A method for detecting building changes in remote sensing images, characterized in that, The application relates to a building change detection model, comprising two inputs for inputting a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputting a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively.

2. The method of claim 1, wherein, The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. where Δp n is a learnable parameter representing the offset prediction value, P0 represents the coordinates of the center of the convolution kernel relative to the upper left corner of the input feature map, P n is the offset of the convolution kernel element relative to the center of the convolution kernel, w is the value of the corresponding position of the convolution kernel, x is the input feature map, and y is the convolution output result.

3. The method of claim 1, wherein the method further comprises: The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively.

4. The method of claim 3, wherein the step of detecting the building change comprises the steps of: detecting a building change in the first image; and detecting a building change in the second image. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application relates to a building change detection model, comprising two inputs for inputing a remote sensing image respectively. The application The n feature maps are spliced to form a feature map F0 with n channels.

5. The method for detecting changes in buildings in remote sensing images according to claim 3, characterized in that, The sum of the feature maps F is obtained to obtain a global feature map, comprising: First, use the torch.stack function to stack the input feature map list to generate a new tensor; Then use the torch.sum function to sum the stacked tensor along dimension 0.

6. The method of claim 1, wherein the method further comprises: Each stage of the encoding network includes a 3*3 convolution layer, a RULE function, and a 2*2 max pooling layer connected in turn, and each stage of the decoding network includes an up-sampling layer, a feature offset learning module connected in turn.

7. The method of claim 1, wherein the method further comprises: In the training process of the building change detection model, the loss function used includes a binary cross-entropy loss function and a Dice loss function, and the binary cross-entropy loss function is: where n is the number of samples, y i is the true label, P(y i ) is the true label y i corresponding predicted label value; y i takes the value 0 or 1. The Dice loss function is: where TP is the number of correctly detected changed pixels, FP is the number of falsely detected changed pixels, and FN is the number of falsely detected unchanged pixels.

8. A device for detecting changes in buildings from remote sensing images, characterized in that, Comprising: A model construction unit for constructing a building change detection model, the building change detection model including two inputs for inputting one remote sensing image respectively; A model training unit for training the building change detection model; A building change detection unit for inputting two period original remote sensing image data into the building change detection model to obtain a detection result; The building change detection model comprises: An encoder module including a first encoding network for processing a first period original remote sensing image and a second encoding network for processing a second period original remote sensing image, the first encoding network and the second encoding network being a weight-sharing twin structure, the encoding network including a stages in turn from input to output, a stages for realizing continuous a times down-sampling of the original remote sensing image, and the encoding output feature map output after a times down-sampling of the encoding network being used as a decoding input feature map of a decoder module; A decoder module including a decoding network, the decoding network including a stages in turn from input to output, a stages for realizing continuous a times up-sampling of the decoding input feature map; The stage connection module is further arranged between the encoder module and the decoder module, and is used to transmit the feature map after convolution of each stage of the encoding network to the corresponding stage of the decoding network, and fuse the feature map after up-sampling of the corresponding stage of the decoding network; Wherein, the fusion of the feature map after convolution of each stage of the encoding network and the feature map after up-sampling of the corresponding stage of the decoding network comprises: The two feature maps to be fused have the same width and height; The two feature maps to be fused are spliced and fused and optimized in the feature offset learning module in the decoding network, the feature offset learning module including a preset convolution processing layer, batch normalization and ReLU activation function connected in turn, the preset convolution operation of the preset convolution processing layer including adding a learnable parameter in convolution calculation to adjust the sampling position of the convolution kernel in space according to different positions, different scales and different deformation targets in the feature map, and the learnable parameter is obtained through training.

9. An electronic device, comprising: The electronic device comprises: A processor; A memory for storing processor-executable instructions; Wherein, the processor runs the executable instructions to realize the remote sensing image building change detection method of any one of claims 1-7.

10. A computer readable storage medium having stored thereon computer instructions, wherein, The instructions, when executed by the processor, implement the steps of the remote sensing image building change detection method according to any one of claims 1-7.

Images