Unsupervised change detection method and system for remote sensing images of the same or different sources

Abstract

The present invention discloses an unsupervised change detection method and system for homologous or heterologous remote sensing images. The homologous or heterologous dual-phase images are input into a twin full convolution feature extraction network to obtain feature difference maps of different scales; a feature fusion network based on meta-learning and grouped convolution corresponding to each scale is constructed, and the feature difference maps of different scales are fused to obtain a difference map; an improved objective function is used to optimize the twin full convolution feature extraction network and the feature fusion network based on meta-learning and grouped convolution, and the weight values ​​of the twin full convolution feature extraction network and the feature fusion network are iteratively updated using the Adam optimization method; during the training process, the numerical gap between the changed pixels and the unchanged pixels in the difference map is continuously widened; the trained difference map is binarized through a threshold to obtain a binary change map of the same size as the original image, so that the change detection results of a pair of homologous or heterologous dual-phase remote sensing images can be obtained efficiently and accurately in an unsupervised manner.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and specifically relates to an unsupervised change detection method and system for remote sensing images of the same or different sources, which can be used to accurately detect the changes between two images obtained from the same or different sensors at two different times and generate corresponding change detection graphs. Background Art

[0002] With the increase in the types and number of remote sensing satellites and the improvement of earth observation technology in recent years, people have more opportunities to monitor the changes on the earth's surface from space, and the remote sensing image change detection task has thus emerged. This task can play a vital role in practical applications such as land cover monitoring, disaster management, ecosystem monitoring, and urban planning. In order to achieve change detection, various satellite platforms have provided a large number of multi-temporal remote sensing images for the change detection database in their continuous work, and these images usually have very high spatial resolution. In order to achieve better detection results, general supervised or semi-supervised training methods require a large amount of manually annotated data to ensure the effect of change detection. The manual annotation process is complicated and tedious, so people hope to complete change detection as accurately as possible under unsupervised conditions. However, due to the characteristics of remote sensing images such as rich information and complex spatial details, and the unknowable changes between image pairs, these have brought huge challenges to the change detection of remote sensing images. At the same time, the increase in the types of remote sensing satellites has also put forward new requirements for change detection tasks, namely, change detection tasks for heterogeneous images. When two images of the same location are taken by two different sensors at different times, we call them heterogeneous image pairs. How to remove the interference caused by different sensors and accurately identify the changes between two images brings more challenges to the remote sensing image change detection task.

[0003] Since traditional machine learning methods have good stability and high efficiency, manual features (such as texture features, spectral features, color features, and shape features) are widely used in the field of change detection. Traditional unsupervised change detection methods are usually developed based on difference images, and many practical algorithms have been proposed to obtain useful difference images, such as principal component analysis and slow feature analysis. However, since manual features are difficult to fully describe the information of remote sensing images, the performance of applying traditional machine learning methods in remote sensing image change detection cannot meet our expectations.

[0004] In recent years, with the development of deep learning technology, especially convolutional neural networks, the field of computer vision has entered a new era, and the problem of remote sensing image processing has been greatly solved. Due to the powerful nonlinear fitting ability and hierarchical structure of convolutional neural networks, the learned features can simultaneously obtain high-level semantics and rich spatial context information. A series of remote sensing image change detection methods based on convolutional neural networks have been proposed. The implementation steps of such methods are usually: deep feature extraction and multi-layer feature combination through pre-trained convolutional neural networks, comparison and selection of deep features through certain feature selection strategies, and setting thresholds to generate corresponding binary change maps. However, how to fully explore the multi-scale deep features between image pairs, reasonably select and use the extracted features, and highlight and emphasize the difference information between image pairs are all difficulties faced by the current change detection task. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide an unsupervised change detection method and system for homologous or heterologous remote sensing images in view of the deficiencies in the above-mentioned prior art, construct a multi-scale fully convolutional network to enable the deep feature map to have the spatial relationship between pixels at different scales, improve the full convolutional network's ability to grasp global features and reduce the feature differences between heterologous images through meta-learning, and use grouped convolution to ensure the fusion effect of different features. At the same time, the improved objective function will further highlight the changed area in the difference map, thereby improving the accuracy and recall rate of change detection.

[0006] The present invention adopts the following technical solutions:

[0007] An unsupervised change detection method for homologous or heterologous remote sensing images comprises the following steps:

[0008] S1. Construct a twin full convolution feature extraction network, input the homologous or heterologous dual-phase images into the constructed twin full convolution feature extraction network, extract multi-scale semantic features, and obtain feature difference maps of different scales;

[0009] S2. According to the feature difference maps of different scales obtained in step S1, global features are extracted through a global feature sampling network, and heterogeneous images are mapped into the same feature space; a feature fusion network based on meta-learning and grouped convolution corresponding to each scale is constructed, the feature difference maps of different scales are fused, and a difference map is obtained through a deep change vector analysis method;

[0010] S3. Use the improved objective function to optimize the twin full convolution feature extraction network constructed in step S1 and the feature fusion network based on meta-learning and grouped convolution in step S2, and use the Adam optimization method to iteratively update the weight values ​​of the twin full convolution feature extraction network and the feature fusion network; during the training process, continuously increase the numerical gap between the changed pixels and the unchanged pixels in the difference map obtained in step S2;

[0011] S4, binarize the difference map trained in step S3 through a threshold to obtain a binary change map of the same size as the original image, and complete image change detection. 2. The method according to claim 1 is characterized in that in step S1, the twin full convolution feature extraction network is a set of dual-branch weight-sharing full convolution networks, each of which is composed of a VGG16 model consisting of five convolution modules connected in series.

[0012] Specifically, in step S1, the input of the homologous or heterologous dual-phase image is specifically:

[0013] The homologous image pairs are resized to 640×640, the bi-phase images are normalized by subtracting the mean of each pixel in the homologous image pair and dividing it by the standard deviation, and the preprocessing method is used to complete the registration of the input homologous image pair;

[0014] The size of the heterogeneous image pair is resized to 640×640, and the dual-phase image is standardized by subtracting the mean of each pixel in the heterogeneous image pair and dividing it by the standard deviation, and the two images are registered using the preprocessing method;

[0015] The processed homologous or heterologous images are input into the twin fully convolutional feature extraction network and feature fusion network to obtain the corresponding feature difference map.

[0016] Specifically, in step S2, the feature fusion network based on meta-learning and grouped convolution includes a global feature sampling network, a feature upsampling model based on a meta-learning module, and a feature fusion model based on a grouped convolution module; the global feature sampling network is a U-shaped network with two branches and non-shared weights, and a skip connection is used between the corresponding layers of the encoder and decoder of the U-shaped network; the feature upsampling module based on the meta-learning module includes a position encoding process and an inverted residual block; the feature fusion module based on grouped convolution is composed of four grouped convolution modules with different convolution kernel sizes connected in parallel.

[0017] Specifically, in step S3, the feature difference maps of different scales obtained in step S2 are passed through a deep change vector analysis network to obtain the change amplitude ρ of the corresponding position of each pixel, and the network parameters of the twin full convolution feature extraction network and the feature fusion network based on meta-learning and group convolution are updated using an improved loss function, so that the change amplitude ρ continuously approaches 0 or 1, and the optimal difference map is obtained.

[0018] Specifically, in step S3, the Adam optimization method is used to iteratively update the network weight value as follows:

[0019] A pair of homologous or heterologous dual-phase images are input into the twin full convolution feature extraction network and the feature fusion network based on meta-learning and grouped convolution, and the weight values ​​of the twin full convolution feature extraction network and the feature fusion network based on meta-learning and grouped convolution are updated using weights; the homologous or heterologous dual-phase images are repeatedly input into the twin full convolution feature extraction network and the feature fusion network based on meta-learning and grouped convolution, and the loss function loss value after the weight value is updated is updated.

[0020] Furthermore, the loss function loss is as follows:

[0021]

[0022] Among them, τ local (i, j) is the local adaptive threshold, H is the length of the difference map, W is the width of the difference map, ρ i,j is the numerical value of the difference map at position (i, j), that is, the magnitude of the change in the pixel at position (i, j).

[0023] Furthermore, the updated weight value W new for:

[0024]

[0025] Among them, W is the initial weight value of the multi-scale graph convolutional neural network, L is the learning rate of the multi-scale graph convolutional neural network training, Represents the partial derivative operation.

[0026] Specifically, in step S4, the binarization process of the difference map is expressed as:

[0027]

[0028] Among them, τ N is the threshold used in the loss function to distinguish between changed and unchanged areas in the last training process, and CM(x,y) is the expected binary change map.

[0029] Another technical solution of the present invention is an unsupervised change detection system for homologous or heterologous remote sensing images, comprising:

[0030] Feature module: construct a twin full convolution feature extraction network, input the homologous or heterologous dual-phase images into the constructed twin full convolution feature extraction network, extract multi-scale semantic features, and obtain feature difference maps of different scales;

[0031] The fusion module extracts global features through the global feature sampling network based on the feature difference maps of different scales obtained by the feature module, and maps the heterogeneous images into the same feature space; constructs a feature fusion network based on meta-learning and grouped convolution corresponding to each scale, fuses the feature difference maps of different scales, and obtains the difference map through the deep change vector analysis method;

[0032] The optimization module uses an improved objective function to optimize the twin full convolution feature extraction network constructed by the feature module and the feature fusion network based on meta-learning and grouped convolution in the fusion module. The Adam optimization method is used to iteratively update the weight values ​​of the twin full convolution feature extraction network and the feature fusion network. During the training process, the numerical gap between the changed pixels and the unchanged pixels in the difference map obtained by the fusion module is continuously widened.

[0033] The detection module performs a binarization operation on the difference map trained by the optimization module through a threshold, obtains a binary change map of the same size as the original image, and completes image change detection.

[0034] Compared with the prior art, the present invention has at least the following beneficial effects:

[0035] The present invention discloses an unsupervised change detection method for homologous or heterologous remote sensing images. For homologous and heterologous images, the method can fully mine and retain the depth features of remote sensing images, effectively fuse multi-scale features, and organically combine the spatial and spectral features in the feature map. Finally, the improved objective function effectively highlights the change area in the difference map, and completes accurate change detection. For heterologous image pairs, the method can also effectively reduce the data differences caused by different sensors, thereby effectively ensuring the change detection effect of heterologous images.

[0036] Furthermore, the present invention uses a twin fully convolutional feature extraction network as the feature extraction unit of the method, which can extract multi-scale depth feature information of the dual-phase image to be detected through several layers of convolution, thereby obtaining a difference feature map containing feature information of different scales, providing effective data support for the subsequent change detection task.

[0037] Furthermore, the homologous image pair used in the present invention adopts a pair of dual-phase multi-spectral high-resolution satellite images of the Montpellier area provided in the Onera Satellite Change Detection public data set. The heterogeneous image pair used in the present invention adopts a pair of dual-phase synthetic aperture radar (SAR) / optical satellite images taken in Wuhan, China. Adjusting its size to 640×640 ensures the consistency of the size of subsequent convolution and deconvolution operations. The standardization operation makes the image pair pixel value distribution conform to the data distribution law, making it easier to obtain the generalization effect after training.

[0038] Further, the upsampling module based on meta-learning can be used to guide the generation of convolution weights in the upsampling convolution layer by fully fusing the global features of the image, thereby effectively avoiding information loss during the upsampling process. The feature fusion module based on grouped convolution can fully fuse the upsampled features and the original features at this scale, further avoiding feature loss during the upsampling process of the full convolution network. At the same time, the feature fusion module further utilizes the relationship between the channels of the feature map, introduces the spectral relationship between the feature maps for the generation of the final difference map, and improves the final change detection effect. In the upsampling module based on meta-learning of the present invention, a global feature sampling network is used to fuse the global features in the image, and it will also be used to reduce the feature differences between heterogeneous image pairs, thereby effectively ensuring the detection accuracy of heterogeneous images. In the upsampling module based on meta-learning, an inverted residual block is also used, which can allocate the global features obtained for each meta-block generated by position in the residual block, and through the dynamic block convolution function, it is used to guide the generation of weight parameters, so that the difference feature map can better grasp the relationship between each pixel in the difference feature map space, and better realize the upsampling of the deep network. In the feature fusion module based on grouped convolution, grouped convolution is used by adopting grouped convolutions with different convolution kernel sizes to better complete the feature fusion between different feature difference maps and capture the features between the difference feature maps.

[0039] Furthermore, a deep change vector analysis network is used to convert difference feature maps of different scales into difference maps that can represent the change amplitude of each pixel corresponding to the position, so that the difference map can be directly operated to obtain the required change detection map. A new objective function for change detection is proposed, which emphasizes the intra-class similarity between the changed area and the unchanged area, continuously widens the inter-class difference between the changed area and the unchanged area, thereby highlighting the changed area, ensuring the final effect of change detection, and improving the robustness of the detection process.

[0040] Furthermore, the Adam optimization algorithm is used to update the network weight values, which comprehensively considers the first-order moment estimation and the second-order moment estimation of the gradient, and is more suitable for problems where the gradient is sparse or there is a lot of noise in the gradient and unstable objective functions, and can play a better role in the unsupervised mode of the present invention.

[0041] Furthermore, the improved loss function is used to continuously update the parameters to obtain the weight value W that can better generate the image difference map new , thus ensuring the final change detection accuracy.

[0042] Furthermore, the present invention uses the threshold used by the last improved loss function to perform the binarization process of the difference map. Due to the updatability of the improved loss function and its threshold, the present invention can better distinguish the changed area from the unchanged area, and obtain the final binary change map, which ensures that the binary change map can well reflect the changes between the two images of the dual-phase image pair.

[0043] In summary, the present invention can efficiently and accurately obtain the change detection results of a pair of homologous or heterologous dual-phase remote sensing images in an unsupervised manner.

[0044] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 is a flow chart of the present invention;

[0046] Figure 2 Schematic diagram of the meta-learning upsampling module of the present invention;

[0047] Figure 3 Schematic diagram of the grouped convolution module of the present invention;

[0048] Figure 4 Schematic diagram of the overall change detection network of the present invention. DETAILED DESCRIPTION

[0049] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0050] In the description of the present invention, it should be understood that the terms “include” and “comprises” indicate the presence of described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or collections thereof.

[0051] It should also be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0052] It should be further understood that the term "and / or" used in the present description and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0053] Various structural schematic diagrams of the embodiments disclosed in the present invention are shown in the accompanying drawings. These figures are not drawn to scale, and some details are magnified and some details may be omitted for the purpose of clear expression. The shapes of various regions and layers shown in the figures and the relative sizes and positional relationships therebetween are only exemplary, and may deviate in practice due to manufacturing tolerances or technical limitations, and those skilled in the art may additionally design regions / layers with different shapes, sizes, and relative positions according to actual needs.

[0054] The present invention provides an unsupervised change detection method for homologous or heterologous remote sensing images, which includes inputting a homologous or heterologous dual-time phase image pair; constructing a full convolution feature extraction network; constructing a feature fusion network based on meta-learning and grouped convolution; completing the training of the overall network through an improved objective function; and finally generating a binary change map corresponding to the input image pair.

[0055] The present invention utilizes a meta-learning module to fuse global features in the image and reduce feature differences between heterogeneous image pairs, and utilizes a feature fusion module based on grouped convolution to fully fuse features of different scales and introduce inter-spectral relationships into the difference feature map. The present invention also uses an improved objective function to further highlight the sample features of the change area and the unchanged area, thereby ensuring the reliability of the training process. The above modules work together to make the final detection result of the method highly robust and accurate, and can be well applied to homologous or heterologous change detection tasks.

[0056] See also Figure 1 The present invention provides an unsupervised change detection method for homologous or heterologous remote sensing images, comprising the following steps:

[0057] S1. Build a twin fully convolutional feature extraction network;

[0058] The twin fully convolutional feature extraction network is a set of dual-branch weight-sharing fully convolutional networks, each of which is composed of a VGG16 model consisting of five convolutional modules connected in series.

[0059] In the VGG16 model, the convolution kernel size is set to 3×3 pixels, the step size is set to 1×1 pixels, and the padding is set to 1 pixel. The model uses the weights pre-trained on the building extraction dataset as the initial value.

[0060] In the network parameters of VGG16, the input feature maps of the second, third, fourth, and fifth convolution modules are set to 64, 128, 256, and 512, respectively, and the output feature maps are set to 128, 256, 512, and 512, respectively. Each module is connected through an average pooling layer to reduce the size of the feature map and increase the receptive field of the convolution kernel, so that the network can extract richer multi-scale information.

[0061] Input a pair of homologous or heterologous dual-phase images;

[0062] In order to effectively verify the effectiveness of the invented unsupervised change detection method, homologous and heterologous dual-phase images provided in the public dataset will be used. For the image pairs used, after certain preprocessing, they are respectively input into the twin full convolution feature extraction network built in the previous article and the feature fusion network based on meta-learning and group convolution. Through continuous training, the change results of this pair of image pairs can be better detected.

[0063] S101, homologous image pair data uses dual-phase images of the Montpellier area provided in the Onera Satellite Change Detection (OSCD) public dataset, which is a pair of multispectral satellite images taken by the Sentinel-2 satellite in the Montpellier area on August 12, 2015 and October 30, 2017, with a spatial resolution of 10 meters / pixel and a size of 451×426 pixels. The dataset provides a reference image that represents the changes between the two time periods.

[0064] This method uses four spectral bands of red, green, blue and near-infrared in the multi-spectrum as research bands.

[0065] The pair of images were resized to 640×640, and the dual-phase images were normalized by subtracting the mean of each pixel in the image pair and dividing it by the standard deviation. The relevant preprocessing methods were used to complete the registration of the input homologous image pair.

[0066] S102, heterogeneous image pair data uses a pair of synthetic aperture radar (SAR) / optical satellite images taken in Wuhan, China. The SAR image was taken by Radarsat-2 in C band in June 2008, and the optical image was obtained by Google Earth in November 2011. The pixel size of both images is 495×503. In this dataset, the main changes in the scene are changes in buildings and roads. This dataset provides a reference image that represents the changes between two time periods.

[0067] The pair of images was resized to 640×640, and the dual-phase images were normalized by subtracting the mean of each pixel in the image pair and dividing it by the standard deviation. The relevant preprocessing methods were used to complete the registration of the input heterogeneous image pair.

[0068] The processed homologous or heterologous images are input into the twin fully convolutional feature extraction network built by S1 and S2 and the feature fusion network based on meta-learning and group convolution to obtain the corresponding difference map.

[0069] S2. Construct a feature fusion network based on meta-learning and group convolution;

[0070] Construct a feature fusion network that can not only eliminate the feature differences of heterogeneous images, but also fully integrate features of different scales. In order to achieve this goal, the present invention constructs a feature fusion network based on meta-learning and grouped convolution. The feature fusion network consists of three parts: a global feature sampling network, a feature upsampling model based on a meta-learning module, and a feature fusion model based on a grouped convolution module. Figure 3 shown.

[0071] First, the feature maps output by the two branches of the twin fully convolutional feature extraction network will be respectively input into the global feature sampling network with the same structure but independent parameters to grasp the global features of the images in the two time periods. At the same time, the global features extracted by the global feature sampling network are used to guide the generation of feature weights in the sampling stage of the twin fully convolutional feature extraction network. In addition, for heterogeneous images, the twin fully convolutional feature extraction network and the global feature sampling network are further preprocessed as input to the dual-phase images, and finally the two images are classified by the classifier. In this way, the feature differences between the two images are eliminated, thereby obtaining features that can be used for change detection in the same feature space. (Here is an introduction to the global feature sampling network)

[0072] Next, for the feature maps of two time periods obtained at different convolutional module layers in the twin full convolutional feature extraction network, the feature difference map of the corresponding layer is obtained by pixel-by-pixel difference operation. Under the guidance of the global feature sampling network, the weight parameters corresponding to different positions will be provided for the difference feature map obtained at each layer of the twin full convolutional feature extraction network, and the difference feature map containing the global feature and position relationship will be obtained through the meta-learning module.

[0073] The upsampled difference feature map is connected to the difference map of the current layer. Through the grouped convolution module, grouped convolutions of different scales can not only grasp the characteristics of the changing regions of different forms, but also consider the relationship between feature channels, giving the fused difference features a spectral relationship. (Here is an introduction to the feature fusion model based on the grouped convolution module)

[0074] Through feature upsampling based on the meta-learning module and feature fusion based on the grouped convolution module, five difference feature maps corresponding to the five convolution module scales of the twin full convolution feature extraction network will be obtained. By inputting them into the deep change vector analysis module, the final difference map can be obtained.

[0075] Next, each module in the feature fusion network will be described in detail in conjunction with the structure diagram.

[0076] See also Figure 2 and Figure 4 , further describe the structure in the global feature sampling network and meta-learning module.

[0077] First, we introduce the global feature sampling network, such as Figure 4 As shown in Figure 2, the global feature sampling network is similar to a U-shaped network, and skip connections are used between the corresponding layers of the encoder and decoder of the U-shaped network.

[0078] In the encoder of the global feature sampling network, two convolutional layers are used to obtain feature maps of two different scales, and global average pooling is used on the last layer of the decoder to obtain a global feature of size 1. The first convolutional layer contains a convolution operation and a downsampling operation. In the convolution operation, the convolution kernel size is set to 1×1 pixels, the stride is set to 1×1 pixels, the padding is set to 0 pixels, and the downsampling uses an average pooling layer. The second convolutional layer contains a convolution operation, in which the convolution kernel size is set to 2×2 pixels, the stride is set to 2×2 pixels, and the padding is set to 0 pixels. The number of channels of the feature map of each layer in the encoder is the same as that of the input feature map, which is 512.

[0079] In the decoder of the global feature sampling network, bilinear interpolation is used for upsampling of global features to obtain a feature map of the same size as the second convolutional layer. After cascading the two, they are fused through a convolution operation with a kernel size of 1×1 pixels, a step size of 1×1 pixels, and a padding of 0 pixels. After that, two upsampling operations are performed, which are completed by deconvolution operations. After each upsampling, it is cascaded with the encoder output features of the corresponding size, and then fused through a convolution operation with a kernel size of 1×1 pixels, a step size of 1×1 pixels, and a padding of 0 pixels. Finally, the two global feature extraction networks with the same structure will obtain a feature map with the same output size as the 5th layer of the twin fully convolutional feature extraction network, but with global features fused. The difference between these two feature maps is obtained to obtain a feature difference map containing global features.

[0080] For heterogeneous image pairs, the twin fully convolutional feature extraction network and the encoder of the global feature extraction network are pre-trained to eliminate the feature differences between heterogeneous images. The global feature of size 1 obtained after global average pooling in the encoder of the global feature extraction network is classified by a convolution operation with a convolution kernel size of 1×1 pixels, a step size of 1×1 pixels, and a padding of 0 pixels. The number of categories is 2, which is used to identify whether the feature maps of two heterogeneous images are in the same feature space.

[0081] See also Figure 2 ,Next, we will introduce the structure of the meta-learning based upsampling module.

[0082] The main process of the meta-learning-based upsampling module includes the position encoding process and the inverted residual block. It mainly uses the feature map containing global information obtained by the global feature sampling network to give global features to each layer of difference features and guide the upsampling process.

[0083] First, for each layer of difference features, it includes a position encoding process to ensure that different weights can be assigned to different positions in the subsequent links. The position encoding process is shown in the following formula:

[0084]

[0085] in, Represents the position code of the difference feature map at (i, j), H and W are the length and width of the current difference feature map respectively. Next, the position code and the difference feature map will be jointly input into the subsequent inverted residual block.

[0086] The inverted residual block is a convolution structure consisting of two groups of regular convolutions and one deep convolution. First, a convolution operation with a kernel size of 1×1 pixels, a stride of 1×1 pixels, and a padding of 0 pixels is performed to increase the dimension of the input feature map, followed by a batch normalization layer and a ReLU activation function; next, a deep convolution operation with a kernel size of 3×3 pixels, a stride of 1×1 pixels, and a padding of 1 pixel is performed to grasp more comprehensive feature information without excessively increasing network parameters, followed by a batch normalization layer and a ReLU activation function; finally, a convolution operation with a kernel size of 1×1 pixels, a stride of 1×1 pixels, and a padding of 0 pixels is performed to restore the dimension of the input feature map to the dimension of the input feature map.

[0087] In the inverted residual block used in the present invention, the input feature dimensions are 64, 128, 256, 512, 512 from the 1st layer to the 5th layer, respectively, and the increased feature dimensions are 128, 256, 512, 1024, 1024, respectively.

[0088] The weights of the convolution in the inverted residual block are generated under the guidance of the feature map containing global information. This process is the meta-learning process. First, the number of channels of the feature map containing global information needs to be divided into several parts. This process is allocated according to the number of weights required for each meta-block (there are 5 blocks in the present invention). Next, the global features obtained by allocating each meta-block are used to guide the generation of weight parameters through the dynamic block convolution function. The operation process of the dynamic block convolution function is shown in the following formula:

[0089] O i,j =X i,j *θ i,j

[0090] Where X i,j represents the difference feature map of the input, θ i,j is the global feature, O i,j The final weight for the inverted residual block is . Due to the position encoding obtained previously, the corresponding multiplication of each position can be achieved during the dynamic block convolution process, thereby giving the final weights a positional relationship, and also enabling the feature map after the inverted residual block to better grasp the relationship between each pixel in the difference feature map space.

[0091] See also Figure 3 , further describe the structure in the grouped convolution module.

[0092] The grouped convolution module consists of 4 groups of grouped convolutions with different convolution kernel sizes. The main structure includes a feature cascade part and a grouped convolution feature fusion part. The feature cascade part fuses the upsampled difference feature map with the feature map of the current layer, that is, connects channel by channel to obtain a feature map with 2C channels. Next, the grouped convolution feature fusion part is input, which consists of four independent grouped convolutions. Each grouped convolution is divided into C groups, which means that there is a dedicated convolution kernel for each group of channels in the feature map (two channels come from two different feature maps), so as to better capture the features between feature maps. In addition, since the location and range of the change area in the change detection task vary greatly, in order to better grasp the change areas of different sizes and shapes, the sizes of the convolution kernels in the grouped convolution are also different. The sizes of the four convolution kernels are 1×1, 3×1, 1×3 and 3×3, respectively, and finally 4 feature maps with the same size as the input feature map and 2C channels are obtained. Next, the feature maps obtained by the four group convolutions are cascaded, and through a convolution operation with a convolution kernel size of 1×1, a fused feature map with a channel number of C is finally obtained, completing the channel fusion operation of the corresponding scale feature difference map. C is 64, 128, 256, 512, and 512 in different layers from the 1st layer to the 5th layer.

[0093] S3. Improve the objective function to complete network optimization

[0094] In order to optimize the twin fully convolutional feature extraction network and the global feature sampling network built in step S1 and step S2, the deep change vector analysis method is first used to generate a difference map with a numerical range of [0,1] from the feature difference map. Then, the loss function designed by the present invention and the commonly used Adam optimization algorithm are used to optimize them, so as to maximize the numerical difference between the changed pixels and the unchanged pixels in the difference map, so as to better complete the generation of the binary change map in S5.

[0095] Through the built twin full convolution feature extraction network and global feature sampling network, the homologous or heterologous image pairs will obtain difference feature maps containing 5 different scales. Through the deep change vector analysis network, the change amplitude of each pixel corresponding position can be obtained, as shown in the following formula:

[0096]

[0097] Where ρ represents the change amplitude of the characteristic graph. A large ρ value represents a high degree of change, and a small ρ value represents a low degree of change. d This is the difference feature map obtained by the network. Through this operation, a change map with a dimension of 1 will be obtained, and the value of each pixel in the map represents the magnitude of the change at that position.

[0098] For general methods, after obtaining the change map, a specific threshold selection method can be used to divide the degree of change into two categories, and a binary change map can be obtained to complete the change detection. In the present invention, in order to further highlight the change area and obtain a more accurate regional boundary, an improved loss function is proposed, so that the twin full convolution feature extraction network built in step S1 and step S2 and the feature fusion network based on meta-learning and grouped convolution can be continuously updated during the training process to better grasp the characteristics of the input image pair.

[0099] The formula of the loss function is as follows:

[0100]

[0101] Among them, H is the length of the difference map, W is the width of the difference map, and ρ i,j is the numerical value of the difference map at position (i, j), that is, the change amplitude of the pixel at position (i, j), τ local (i, j) is the local adaptive threshold, which can capture strong local changes in the image and highlight the boundary of the change map. Low-pass filtering is used to determine the local adaptive boundary, and Gaussian filtering is selected to determine the local adaptive threshold, that is, to generate a context-dependent threshold for each pixel by generating a weighted average of the neighborhood of each pixel.

[0102] Next, use the Adam optimization algorithm to iteratively update the network weight value. The steps are as follows:

[0103] In the first step, a pair of homologous or heterologous dual-phase images are input into the constructed twin full convolution feature extraction network and the feature fusion network based on meta-learning and grouped convolution, and the weights are used to update the weight values ​​of the twin full convolution feature extraction network and the feature fusion network based on meta-learning and grouped convolution:

[0104]

[0105] Among them, W new is the updated weight value, W is the initial weight value of the multi-scale graph convolutional neural network, L is the learning rate of the multi-scale graph convolutional neural network training, and the learning rate range is [0.001-0.00001], × represents the multiplication operation, represents the partial derivative operation;

[0106] In the second step, the homologous or heterologous dual-phase image pairs are repeatedly input into the twin fully convolutional feature extraction network and the feature fusion network based on meta-learning and group convolution, and the loss function loss value after the weight value is updated is updated.

[0107] S5. Generate binary change map

[0108] Through the continuous optimization of the improved objective function proposed in step S4, the difference map obtained by the depth change vector analysis method will be continuously updated, and finally a difference map with the same size as the input image will be obtained. The difference map can be binarized by the threshold used by the last improved loss function. The binarization process is expressed as:

[0109]

[0110] where τ N is the threshold used in the loss function to distinguish between changed and unchanged areas in the last training process. CM(x,y) is the expected binary change map, where 0 represents unchanged areas and 1 represents changed areas. Through the above process, a binary change map of size 640×640 can be obtained, which is restored to the original size of the image in the dataset to obtain the final binary change map.

[0111] In yet another embodiment of the present invention, there is provided an unsupervised change detection system for homologous or heterologous remote sensing images. The system can be used to implement the above-mentioned unsupervised change detection method for homologous or heterologous remote sensing images. Specifically, the unsupervised change detection system for homologous or heterologous remote sensing images includes a feature module, a fusion module, an optimization module and a detection module.

[0112] Among them, the feature module constructs a twin full convolution feature extraction network, inputs the homologous or heterologous dual-phase images into the constructed twin full convolution feature extraction network, extracts multi-scale semantic features, and obtains feature difference maps of different scales;

[0113] The fusion module extracts global features through the global feature sampling network based on the feature difference maps of different scales obtained by the feature module, and maps the heterogeneous images into the same feature space; constructs a feature fusion network based on meta-learning and grouped convolution corresponding to each scale, fuses the feature difference maps of different scales, and obtains the difference map through the deep change vector analysis method;

[0114] The optimization module uses an improved objective function to optimize the twin full convolution feature extraction network constructed by the feature module and the feature fusion network based on meta-learning and grouped convolution in the fusion module. The Adam optimization method is used to iteratively update the weight values ​​of the twin full convolution feature extraction network and the feature fusion network. During the training process, the numerical gap between the changed pixels and the unchanged pixels in the difference map obtained by the fusion module is continuously widened.

[0115] The detection module performs a binarization operation on the difference map trained by the optimization module through a threshold, obtains a binary change map of the same size as the original image, and completes image change detection.

[0116] In another embodiment of the present invention, a terminal device is provided, which includes a processor and a memory, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, which is suitable for implementing one or more instructions, and is specifically suitable for loading and executing one or more instructions to implement the corresponding method flow or corresponding function; the processor described in the embodiment of the present invention can be used for the operation of the unsupervised change detection method for homologous or heterologous remote sensing images, including:

[0117] A twin full convolutional feature extraction network is constructed, and the homologous or heterologous dual-phase images are respectively input into the constructed twin full convolutional feature extraction network to extract multi-scale semantic features and obtain feature difference maps of different scales; according to the feature difference maps of different scales, global features are extracted through the global feature sampling network, and the heterologous images are mapped to the same feature space; a feature fusion network based on meta-learning and grouped convolution corresponding to each scale is constructed, and the feature difference maps of different scales are fused, and the difference map is obtained through the deep change vector analysis method; the twin full convolutional feature extraction network and the feature fusion network based on meta-learning and grouped convolution are optimized using the improved objective function, and the weight values ​​of the twin full convolutional feature extraction network and the feature fusion network are iteratively updated using the Adam optimization method; during the training process, the numerical gap between the changed pixels and the unchanged pixels in the difference map is continuously widened; the trained difference map is binarized through a threshold to obtain a binary change map of the same size as the original image, and image change detection is completed.

[0118] In another embodiment of the present invention, the present invention further provides a storage medium, specifically a computer-readable storage medium (Memory), which is a memory device in a terminal device for storing programs and data. It can be understood that the computer-readable storage medium here can include both the built-in storage medium in the terminal device and the extended storage medium supported by the terminal device. The computer-readable storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space, and these instructions can be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here can be a high-speed RAM memory, or a non-volatile memory (non-volatile memory), such as at least one disk storage.

[0119] The processor may load and execute one or more instructions stored in the computer-readable storage medium to implement the corresponding steps of the unsupervised change detection method for homologous or heterologous remote sensing images in the above embodiment; the processor may load and execute the following steps:

[0120] A twin full convolutional feature extraction network is constructed, and the homologous or heterologous dual-phase images are respectively input into the constructed twin full convolutional feature extraction network to extract multi-scale semantic features and obtain feature difference maps of different scales; according to the feature difference maps of different scales, global features are extracted through the global feature sampling network, and the heterologous images are mapped to the same feature space; a feature fusion network based on meta-learning and grouped convolution corresponding to each scale is constructed, and the feature difference maps of different scales are fused, and the difference map is obtained through the deep change vector analysis method; the twin full convolutional feature extraction network and the feature fusion network based on meta-learning and grouped convolution are optimized using the improved objective function, and the weight values ​​of the twin full convolutional feature extraction network and the feature fusion network are iteratively updated using the Adam optimization method; during the training process, the numerical gap between the changed pixels and the unchanged pixels in the difference map is continuously widened; the trained difference map is binarized through a threshold to obtain a binary change map of the same size as the original image, and image change detection is completed.

[0121] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. The components of the embodiments of the present invention described and shown in the drawings here can usually be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0122] The effects of the present invention are further described below in conjunction with simulation experiments.

[0123] 1. Simulation conditions

[0124] The hardware platform of the simulation experiment of the present invention is: CPU is Intel (R) Core (TM) i7-8700X, the main frequency is 3.2GHz, the memory is 64GB, and the GPU is NVIDIA 2080Ti.

[0125] The software platform of the simulation experiment of the present invention is: Ubuntu operating system and python3.6.

[0126] 2. Simulation content and result analysis

[0127] The simulation experiment of the present invention will be used to measure the detection ability of the unsupervised remote sensing image change detection method based on meta-learning and group convolution constructed by the present invention for homologous and heterologous images.

[0128] For the homologous dual-phase image pairs, the method proposed in the present invention and five existing technologies (robust change vector analysis method, principal component analysis network method, symmetrical coupled convolutional network method, deep change vector analysis method, deep belief network method) are respectively unsupervised trained using training image pairs. After the training is completed, the pair of images is subjected to change detection to obtain a binary change map for this set of images.

[0129] The dual-phase image pair used for the homologous images in the simulation experiment of the present invention comes from a group of multispectral satellite images of the Montpellier area in the Onera Satellite Change Detection (OSCD) data set. This group of multispectral satellite images was taken by the Sentinel-2 satellite in the Montpellier area on August 12, 2015 and October 30, 2017, respectively, with a spatial resolution of 10 meters / pixel and a size of 451×426 pixels. There are images of 13 spectral channels in the data set. In this experiment, we selected four spectral bands, namely red, green, blue and near-infrared, as the research bands. This pair of images provides a reference image representing the changes between the two time periods, but it does not participate in the training process, and only participates in the change detection results of each method in the final evaluation experiment.

[0130] For heterogeneous dual-phase image pairs, the method proposed in the present invention and three existing technologies (multimodal Markov model method, symmetric coupled convolutional network method, and patch similarity graph matrix method) are used to perform unsupervised training using training image pairs. After the training is completed, the pair of images is subjected to change detection to obtain a binary change map for this set of images.

[0131] The dual-phase image pair used in the simulation experiment of the present invention for heterogeneous images comes from a pair of synthetic aperture radar (SAR) / optical satellite images taken in Wuhan, China. The SAR image was taken by Radarsat-2 in C band in June 2008, and the optical image is the data of red, green and blue bands obtained by Google Earth in November 2011. The pixel size of both images is 495×503. In this data set, the main changes in the scene are the changes in buildings and roads. This data set provides a reference image representing the changes between the two time periods, but it does not participate in the training process, and only participates in the change detection results of each method in the final evaluation experiment.

[0132] In the simulation experiment of homologous image pairs, the five existing technologies used are:

[0133] The prior art robust change vector analysis method refers to the remote sensing image change detection method proposed by Thonfeld et al. in their paper “Robust change vector analysis (RCVA) for multi-sensor very high resolution optical satellites” (Int. J. Appl. Earth Observ. Geoinf., 2016). This method uses neighborhood information between pixels for unsupervised change detection, referred to as the robust change vector analysis method.

[0134] The prior art principal component analysis network method refers to the remote sensing image change detection method proposed by Gao et al. in their published paper “Automatic change detection in synthetic aperture radar images based on PCANet” (IEEE Geosci. Remote Sens. Lett., 2016), referred to as the principal component analysis network method.

[0135] The prior art symmetric coupled convolutional network method refers to the remote sensing image change detection method proposed by Liu et al. in their published paper "A deep convolutional coupling network for change detection based on heterogeneous optical and radar images" (IEEE Trans. Neural Netw. Learn. Syst., 2018). This method is a method based on deep feature changes, referred to as the symmetric convolutional coupled network method.

[0136] The prior art deep change vector analysis method refers to the remote sensing image change detection method proposed by Saha et al. in their paper "Unsupervised deep change vector analysis for multiple-change detection in VHR images" (IEEE Trans. Geosci. Remote Sens., 2019). This method is based on a deep learning method and can detect changes in image pairs by performing direct deep feature comparison based on the results of change vector analysis. It is referred to as the deep change vector analysis method.

[0137] The prior art deep belief network method refers to the paper "Change detection in synthetic aperture radar images based on deep neural networks" (IEEE Trans. Neural Netw. Learn. Syst., 2016) published by Gong et al., which creates a model that can predict the label of each spatial position by learning the concepts of pixel change and pixel invariance from the initial classification results to complete unsupervised change detection, referred to as the deep belief network method.

[0138] Four evaluation indicators (precision, recall, accuracy, and Kappa coefficient) are used to comprehensively compare the change maps of the Montpellier area obtained by the four methods with the reference images provided by the dataset. The following formula is used to calculate precision, recall, accuracy, and Kappa, and the calculation results are plotted in Table 1:

[0139]

[0140]

[0141]

[0142]

[0143] in,

[0144]

[0145] Among them, TP represents the number of pixels in the change map that are correctly classified as the change category, TN represents the number of pixels in the change map that are correctly classified as the unchanged category, FP represents the number of pixels in the change map that are misclassified as the change category, and FN represents the number of pixels in the change map that are misclassified as the unchanged category.

[0146] "The present invention" in Table 1 represents the unsupervised remote sensing image change detection method based on meta-learning and group convolution proposed by the present invention, "RCVA" represents the robust change vector analysis method proposed by Thonfeld et al., "PCANet" represents the principal component analysis network method proposed by Gao et al., "SCCN" represents the symmetric convolutional coupled network method proposed by Liu et al., "DCVA" represents the deep change vector analysis method proposed by Saha et al., and "DBN" represents the deep belief network method proposed by Gong et al.

[0147] Table 1 Performance evaluation table of the present invention and existing unsupervised remote sensing image change detection methods

[0148]

[0149] From Table 1, it can be seen that the binary change map obtained by the present invention after unsupervised learning of the group of dual-phase remote sensing images has an accuracy of 83.56%, a recall of 72.73%, an accuracy of 97.21%, and a Kappa coefficient of 0.7629. These four indicators are all higher than those of the five prior art methods in the simulation experiment, which indicates that this method can achieve higher change detection accuracy.

[0150] In the simulation experiment of heterogeneous image pairs, the three existing technologies used are:

[0151] The prior art multimodal Markov model method refers to the heterogeneous remote sensing image change detection method proposed by Touati et al. in their paper "Multimodal change detection in remote sensing images using an unsupervised pixel pairwise based Markov random field model" (IEEE Trans. Image Process., 2019). This method uses an improved Markov model to complete the change detection of homologous and heterogeneous images, referred to as the multimodal Markov model method.

[0152] The prior art symmetric coupled convolutional network method refers to the heterogeneous remote sensing image change detection method proposed by Liu et al. in their published paper "A deep convolutional coupling network for change detection based on heterogeneous optical and radar images" (IEEE Trans. Neural Netw. Learn. Syst, 2018). This method is a method based on deep feature changes, referred to as the symmetric convolutional coupled network method.

[0153] The prior art based on the patch similarity graph matrix method refers to the heterogeneous remote sensing image change detection method proposed by Sun et al. in their published paper "PatchSimilarity Graph Matrix-Based Unsupervised Remote Sensing Change Detection With Homogeneous and Heterogeneous Sensors" (IEEE Trans. Geosci. Remote Sens., 2020). This method is based on the assumption that a certain part of an image can be represented by its adjacent area, thereby solving the data differences between heterogeneous image pairs. It is referred to as the patch similarity graph matrix method.

[0154] Four evaluation indicators (precision, recall, accuracy, and Kappa coefficient) are used to comprehensively compare the change maps of the Montpellier area obtained by the four methods with the reference images provided by the dataset. The calculation results are plotted in Table 2.

[0155] "The present invention" in Table 2 represents the unsupervised remote sensing image change detection method based on meta-learning and group convolution proposed by the present invention, "M3CD" represents the multimodal Markov model method proposed by Touati et al., "SCCN" represents the symmetric convolutional coupled network method proposed by Liu et al., and "PSGM" represents the patch similarity graph matrix-based method proposed by Sun et al.

[0156] Table 2 Performance evaluation table of the present invention and existing unsupervised heterogeneous remote sensing image change detection methods

[0157]

[0158] From Table 2, it can be seen that the precision of the binary change map obtained by the present invention after unsupervised learning of the group of heterogeneous dual-phase remote sensing images is 75.48%, the recall rate is 71.75%, the accuracy is 95.13%, and the Kappa coefficient is 0.7088. These four indicators are all higher than the three existing technical methods in the simulation experiment, which shows that this method can obtain higher change detection accuracy in the change detection task of heterogeneous dual-phase images.

[0159] In summary, the present invention provides an unsupervised change detection method and system for homologous or heterologous remote sensing images. The twin fully convolutional feature extraction network can fully extract multi-scale and multi-level information. The upsampling network based on meta-learning is used, and the global feature extraction network can grasp the global information of the image pair. At the same time, the feature difference problem between heterologous images can be solved by adding a classifier. At the same time, by fully integrating the global features of the image, it can also be used to guide the generation of convolution weights in the upsampling convolution layer, thereby effectively avoiding information loss in the upsampling process. By using a feature fusion network based on grouped convolution, the upsampled features can be combined with the difference features of the original layer to achieve effective feature fusion. The meta-learning module and the grouped convolution module work together to effectively solve the problem of low detection accuracy caused by incomplete grasp of multi-scale change feature maps in the prior art methods, which is easy to cause content loss. At the same time, the present invention also proposes an improved loss function, which emphasizes the change feature by continuously narrowing the distance between the change and invariant features and their feature centers, and effectively solves the problem of insufficient emphasis on the change area and the inability to accurately detect the irregular change area boundary in the prior art methods. The invention is a very practical unsupervised homologous or heterologous remote sensing image change detection method.

[0160] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0161] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0162] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0163] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0164] The above contents are only for explaining the technical idea of ​​the present invention and cannot be used to limit the protection scope of the present invention. Any changes made on the basis of the technical solution in accordance with the technical idea proposed by the present invention shall fall within the protection scope of the claims of the present invention.

Claims

1. An unsupervised change detection method for homologous or heterologous remote sensing images, characterized in that: The following steps are involved: S1. Construct a twin full convolution feature extraction network, input the homologous or heterologous dual-phase images into the constructed twin full convolution feature extraction network, extract multi-scale semantic features, and obtain feature difference maps of different scales; S2, according to the feature difference maps of different scales obtained in step S1, extract global features through a global feature sampling network, and map heterogeneous images into the same feature space; Construct a feature fusion network based on meta-learning and group convolution corresponding to each scale, fuse the feature difference maps of different scales, and obtain the difference map through the deep change vector analysis method; S3. Use the improved objective function to optimize the twin full convolution feature extraction network constructed in step S1 and the feature fusion network based on meta-learning and grouped convolution in step S2. Use the Adam optimization method to iteratively update the weight values ​​of the twin full convolution feature extraction network and the feature fusion network. During the training process, the numerical gap between the changed pixels and the unchanged pixels in the difference map obtained in step S2 is continuously widened, and the loss function as follows: in, is the local adaptive threshold, is the length of the difference graph, is the width of the difference map, is the numerical value of the difference map at position (i, j), that is, the change amplitude of the pixel at position (i, j); S4, binarize the difference image trained in step S3 through a threshold to obtain a binary change image with the same size as the original image, and complete image change detection.

2. The method according to claim 1, characterized in that In step S1, the twin fully convolutional feature extraction network is a set of dual-branch weight-sharing fully convolutional networks, each of which is composed of a VGG16 model consisting of five convolutional modules connected in series.

3. The method according to claim 1, characterized in that In step S1, the input of the homologous or heterologous dual-phase images is specifically: The homologous image pairs are resized to 640×640, the bi-phase images are normalized by subtracting the mean of each pixel in the homologous image pair and dividing it by the standard deviation, and the preprocessing method is used to complete the registration of the input homologous image pair; The size of the heterogeneous image pair is resized to 640×640, and the dual-phase image is standardized by subtracting the mean of each pixel in the heterogeneous image pair and dividing it by the standard deviation, and the two images are registered using the preprocessing method; The processed homologous or heterologous images are input into the twin fully convolutional feature extraction network and feature fusion network to obtain the corresponding feature difference map.

4. The method according to claim 1, characterized in that In step S2, the feature fusion network based on meta-learning and grouped convolution includes a global feature sampling network, a feature upsampling model based on a meta-learning module, and a feature fusion model based on a grouped convolution module; The global feature sampling network is a dual-branch U-shaped network with non-shared weights, and skip connections are used between the corresponding layers of the encoder and decoder of the U-shaped network; the feature upsampling module based on the meta-learning module includes a position encoding process and an inverted residual block; the feature fusion module based on grouped convolution is composed of four grouped convolution modules with different convolution kernel sizes connected in parallel.

5. The method according to claim 1, characterized in that In step S3, the feature difference maps of different scales obtained in step S2 are passed through the deep change vector analysis network to obtain the change amplitude of each pixel corresponding to the position , the network parameters of the twin full convolution feature extraction network and the feature fusion network based on meta-learning and group convolution are updated using the improved loss function, so that the change range Continuously approaching 0 or 1, we get the optimal difference map.

6. The method according to claim 1, characterized in that In step S3, the Adam optimization method is used to iteratively update the network weight value as follows: A pair of homologous or heterologous dual-phase images are input into the twin full convolution feature extraction network and the feature fusion network based on meta-learning and grouped convolution, and the weight values ​​of the twin full convolution feature extraction network and the feature fusion network based on meta-learning and grouped convolution are updated using the weights; the homologous or heterologous dual-phase images are repeatedly input into the twin full convolution feature extraction network and the feature fusion network based on meta-learning and grouped convolution, and the loss function after the weight value is updated is used. The value is updated.

7. The method according to claim 6, characterized in that Updated weight value for: in, is the initial weight value of the multi-scale graph convolutional neural network, L is the learning rate of the multi-scale graph convolutional neural network training, Represents the partial derivative operation.

8. The method according to claim 1, characterized in that In step S4, the binarization process of the difference map is expressed as: in, To improve the threshold used in the loss function to distinguish between changes and unchanged areas during the last training process, is the expected binary change map.

9. An unsupervised change detection system for homologous or heterologous remote sensing images, characterized in that: include: Feature module: construct a twin full convolution feature extraction network, input the homologous or heterologous dual-phase images into the constructed twin full convolution feature extraction network, extract multi-scale semantic features, and obtain feature difference maps of different scales; The fusion module extracts global features through the global feature sampling network based on the feature difference maps of different scales obtained by the feature module, and maps heterogeneous images into the same feature space; Construct a feature fusion network based on meta-learning and group convolution corresponding to each scale, fuse the feature difference maps of different scales, and obtain the difference map through the deep change vector analysis method; The optimization module uses an improved objective function to optimize the twin full convolution feature extraction network constructed by the feature module and the feature fusion network based on meta-learning and grouped convolution in the fusion module. The Adam optimization method is used to iteratively update the weight values ​​of the twin full convolution feature extraction network and the feature fusion network. During the training process, the numerical gap between the changed pixels and the unchanged pixels in the difference map obtained by the fusion module is continuously widened, and the loss function as follows: in, is the local adaptive threshold, is the length of the difference graph, is the width of the difference map, is the numerical value of the difference map at position (i, j), that is, the change amplitude of the pixel at position (i, j); The detection module performs a binarization operation on the difference map trained by the optimization module through a threshold, obtains a binary change map of the same size as the original image, and completes image change detection.

Images