An object-oriented graph neural network-based unsupervised remote sensing image change detection method

Abstract

This invention discloses an object-oriented graph neural network unsupervised remote sensing image change detection method, relating to the field of remote sensing detection. It aims to improve the accuracy and automation level of change detection in high-resolution remote sensing images by performing image segmentation and feature selection on two temporal phases. It calculates the multi-level weighted difference of image objects in the two phases, uses the EM algorithm and Bayesian minimum error equation to obtain the optimal segmentation threshold, and uses this threshold to initially classify all image objects into two change categories: changed objects and unchanged objects. A certain proportion of the most likely changed and unchanged image objects are adaptively selected as training samples. A skip connection structure is introduced, and the feature fusion radius is adaptively selected, increasing the number of network layers, which is beneficial for high-dimensional feature construction. This method improves change detection efficiency while maintaining accuracy, and has good application value for land cover change detection.

Description

Technical Field

[0001] This invention relates to the field of remote sensing detection, and in particular to an object-oriented graph neural network unsupervised remote sensing image change detection method. Background Technology

[0002] Traditional change detection tasks primarily use image pixels as the smallest processing unit. However, in recent years, with the improvement of image spatial resolution, the spectral differences within similar land features have gradually increased. Automatic change detection techniques based on pixel-level spectral statistics can no longer meet the requirements for extracting change information from current remote sensing images, becoming a major bottleneck restricting the widespread application of high-resolution remote sensing images. The emergence of object-oriented technology has provided a new approach to high-resolution remote sensing image analysis, and the basic unit of change detection has gradually transitioned from pixels to objects. Because object-level change detection algorithms are more advantageous than pixel-level change detection algorithms, they have received widespread attention and development in recent years. Object-oriented change detection methods combine image segmentation methods with the spatial, spectral, and textural information of the image, using image objects as the processing unit for change detection.

[0003] Deep learning methods have been widely applied to change detection in remote sensing images. Remote sensing images have advantages such as wide observation range, short acquisition cycle, high timeliness, and fast update cycle. Remote sensing data is inherently massive, diverse, and multidimensional, giving deep learning a natural advantage in processing it. However, due to the non-Euclidean spatial data characteristics of object-oriented methods in terms of data structure at the processing unit level, they are not suitable for deep learning models such as CNNs that use pixels as processing units. Furthermore, most current deep learning change detection methods rely on labeled samples with supervised information, which requires significant human and material resources. Given the massive amounts of remote sensing data, efficiency must also be considered. Moreover, while manual sample selection can ensure accuracy, it is easily influenced by personal subjectivity, resulting in uneven sample distribution and weak model generalization ability and stability. Summary of the Invention

[0004] To overcome the shortcomings of the existing technologies, this invention proposes an object-oriented graph neural network unsupervised remote sensing image change detection method. By using multi-level segmentation, constructing an adjacency matrix, and adaptive sample selection, it solves the problems of unsuitability of graph neural networks in change detection and insufficient samples.

[0005] To achieve the above objectives, the technical solution adopted is as follows:

[0006] An object-oriented graph neural network-based unsupervised remote sensing image change detection method includes the following steps:

[0007] Step 1: Perform multi-scale, multi-level segmentation and feature selection on the two temporal images to establish a multi-dimensional feature space and obtain multi-temporal feature vectors;

[0008] Step 2: Generate a network graph structure using image segmentation vectors, calculate cosine similarity using node features, and construct a first-order weighted adjacency matrix;

[0009] Step 3: Calculate the weighted difference between the two temporal images and adaptively select the image objects with the highest and lowest probability of change as training samples according to a set ratio.

[0010] Step 4: Network Training and Prediction. Multi-temporal feature vectors are superimposed and imported into a graph convolutional neural network with skip connections. The training samples are used for data training. The classifier predicts all nodes and generates a change result map.

[0011] Step 1 further includes: registering and relative radiometric correction of remote sensing images of different time phases; using a fractal network evolution algorithm to perform multi-scale segmentation of old time phase images, and determining the segmentation scale based on the principle of maximum homogeneity and minimum heterogeneity of segmentation objects; using the segmentation results as the subject layer for segmentation of new time phase images, and controlling and constraining the segmentation results of the subsequent time phase images.

[0012] Based on the bottom-up principle, use respectively The multi-scale segmentation parameters of the group segmented the previous layer image, resulting in a total of The object layer consists of remote sensing image objects of different sizes. Based on the principle of decreasing size, the remote sensing image objects obtained by segmenting with large segmentation scale parameter values ​​completely contain the remote sensing image objects obtained by segmenting with small segmentation scale parameter values.

[0013] Step 2 specifically includes the following steps:

[0014] Step 2.1: Treat remote sensing image objects as nodes, form adjacency edges between adjacent nodes, and generate a first-order adjacency matrix;

[0015] Step 2.2: Calculate the cosine similarity of features between each node and its neighboring nodes, and construct a first-order weighted adjacency matrix, as shown in the following formula:

[0016] (1)

[0017] , Let be the current feature vector index, where: and For the first The first object node and the first Feature vectors of each object node The closer the cosine value is to 1, the smaller the difference between the two image objects; the closer the cosine value is to 0, the greater the difference between the two image objects.

[0018] Step 3 specifically includes the following steps:

[0019] Step 3.1: Extract the spectral, index, geometric, and textural features of the image objects. Use chi-square transform to apply the difference variance of different feature bands as the weights for feature fusion, and calculate the multi-level feature weighted difference degree.

[0020] (2)

[0021] (3)

[0022] in, θ For weight parameters: , These are the multi-scale parameters for object layer 0 and object layer 1, respectively. The number of features of a single-layer image object; For the first Weighted difference of each image, They represent the old and new eras, for 3D feature vectors For the old and new eras The standard deviation of the differential image of each characteristic band;

[0023] Step 3.2: Use the weighted difference of all remote sensing image objects as training samples, obtain the change threshold using the Bayesian threshold determination method based on the EM algorithm, and adaptively select samples.

[0024] The implementation process of adaptive sample selection in step 3.2 is as follows:

[0025] A threshold for changing objects is set, dividing all image objects into changing and non-changing objects. These are then sorted according to weighted differences, and the geometric area of ​​the image objects is used as a constraint, proportionally... Select the top with the highest weighted difference in sequence Each image object is taken as a positive sample, and the image with the lowest weighted difference is selected. Each image object is used as a negative sample.

[0026] The beneficial effects of adopting the above technical solution are as follows:

[0027] I. The remote sensing image change detection method provided by this invention fully utilizes the advantages of graph neural networks in processing non-topological data, and is designed for large-format images to achieve rapid and accurate detection of changed areas.

[0028] Second, to improve the shortcomings of insufficient sample quantity and the need for manual collection in graph neural networks, an unsupervised change detection task is achieved through adaptive sample selection. Attached Figure Description

[0029] Figure 1 A flowchart of an object-oriented graph neural network unsupervised remote sensing image change detection method is provided as an example of the present invention;

[0030] Figure 2 The images provided are multi-temporal remote sensing images, reference change images, and change detection test results using the method described herein, which are examples of this invention. Detailed Implementation

[0031] To facilitate understanding and implementation of the present invention by those skilled in the art, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0032] This embodiment provides an object-oriented graph neural network-based unsupervised remote sensing image change detection method, such as... Figure 1 As shown, it includes the following steps:

[0033] Step 1: Perform multi-scale, multi-level segmentation and feature selection on the two temporal images to establish a multi-dimensional feature space and obtain multi-temporal feature vectors;

[0034] Step 1 further includes: registering and performing relative radiometric correction on remote sensing images of different time phases using ENVI software; performing multi-scale segmentation on old time phase images using the multiresolution segmentation algorithm of eCognition software, determining the segmentation scale parameter as 50, compactness as 0.5, and shape factor as 0.1 using a heuristic method; and using the segmentation result as the subject layer for segmenting new time phase images to control and constrain the segmentation result of the subsequent time phase images.

[0035] Following a bottom-up approach, the previous image layer was segmented using multi-scale segmentation parameters of 100, 300, and 500, resulting in four object layers with remote sensing objects of different sizes. The inclusion relationship between the four object layers at their object boundaries is as follows: That is, a remote sensing image object obtained by segmentation using more than 500 scale segmentation parameters completely contains a remote sensing image object obtained by segmentation using more than 300 scale segmentation parameters; a remote sensing image object obtained by segmentation using more than 300 scale segmentation parameters completely contains a remote sensing image object obtained by segmentation using more than 100 scale segmentation parameters, and so on.

[0036] Step 2: Generate a network graph structure using image segmentation vectors, calculate cosine similarity using node features, and construct a first-order weighted adjacency matrix;

[0037] Step 2 specifically includes the following steps:

[0038] Step 2.1: Treat remote sensing image objects as nodes, form adjacency edges between adjacent nodes, and generate a first-order adjacency matrix;

[0039] Step 2.2: Calculate the cosine similarity of features between each node and its neighboring nodes, and construct a first-order weighted adjacency matrix, as shown in the following formula:

[0040] (4)

[0041] Traditional adjacency matrices only use 0 and 1 to represent the adjacency relationship between adjacent nodes, which cannot reflect the mutual relationships between adjacent nodes. Cosine similarity is used to compensate for this deficiency. , This is the current feature vector index; and For the first The first object node and the first Feature vectors of each object node The closer the cosine value is to 1, the smaller the difference between the two image objects; the closer the cosine value is to 0, the greater the difference between the two image objects. Using the cosine of the vector angle for similarity measurement reflects the geometric characteristics of the pattern, while correcting the problem of inconsistent measurement standards between features, and is more stable.

[0042] Step 3: Calculate the weighted difference between the two temporal images and adaptively select the image objects with the highest and lowest probability of change as training samples according to a set ratio.

[0043] The set of image object differences is defined as follows: , This represents the number of image objects. The classes are divided into changed and unchanged classes. Assume that the conditional density functions of both classes follow a Gaussian distribution. The difference in the mid-pixel speckle approximately follows a mixture Gaussian distribution composed of two sub-Gaussians. The EM algorithm is used to estimate the parameters of the two sub-Gaussian models, obtaining the distribution parameters of the unchanged and changed classes, such as the density function. mean Standard deviation , The change threshold is calculated based on Bayesian minimum error rate theory. :

[0044] (5)

[0045] Using the variance of different characteristic bands as weights for feature fusion, multidimensional features are compressed and fused into a 1D feature space to obtain the weighted difference of image objects. Change detection through chi-square transform feature extraction has been validated. However, due to the lack of consideration for multi-level representation, errors caused by registration problems may lead to high difference in unchanged areas, resulting in incorrect sample selection. To address this, a multi-level weighted difference calculation method is adopted. Since the optimal segmentation scale is difficult to determine, weighted fusion can reduce errors. A multi-level weighted fusion difference calculation method is used. The weighted difference feature space is established by superimposing the bottom object layer and its adjacent upper-level object layer. Multi-scale weighted fusion further reduces the impact of single spectral pseudo-changes caused by registration errors, resulting in higher sample acquisition accuracy. The formula for constructing the multi-level weighted difference is as follows:

[0046] (6)

[0047] (7)

[0048] in, θ For weight parameters: , These are the multi-scale parameters for the 0th and 1st object layers, respectively. The number of features of a single-layer image object; For the first Weighted difference of each image object They represent the old and new eras, for 3D feature vectors For the old and new eras The standard deviation of the differential image of each characteristic band;

[0049] Step 3 specifically includes the following steps:

[0050] Step 3.1: Use eCognition software to extract the spectral, index, geometric, and texture features of the image objects. Spectral features include the mean brightness of each band (blue, green, red, etc.) and the infrared standard deviation. Texture features include the mean and entropy values ​​extracted from the gray-level co-occurrence matrix. Index features include the normalized vegetation index (NVI) and normalized water index for each object layer. Geometric features are the image object area for each object layer. Ten features are selected for each object layer. Note that the spectral, geometric, and texture features of the image objects are already provided in the eCognition attribute table; index features need to be added manually. The formulas for calculating the normalized vegetation index and normalized water index are:

[0051] (8)

[0052] (9)

[0053] in, For the first Mean gray level of the red band of each image object For the first Average gray level of the green band for each image object. For the first Average grayscale value of near-infrared band for each image object.

[0054] The chi-square transform is used to calculate the multi-level feature weighted difference degree by using the difference variance of different feature bands as the weights for feature fusion.

[0055] (10)

[0056] (11)

[0057] in, θ For weight parameters: , These are the multi-scale parameters for the 0th and 1st object layers, respectively. For the first Weighted difference of each image object They represent the old and new eras, for 3D feature vectors For the old and new eras The standard deviation of the differential image of each characteristic band;

[0058] Step 3.2: Use the weighted difference of all image objects as training samples, obtain the change threshold using the Bayesian threshold determination method based on the EM algorithm, and adaptively select samples.

[0059] The implementation process of adaptive sample selection in step 3.2 is as follows:

[0060] A threshold for changing objects is set, dividing all image objects into changing and non-changing objects. These are then sorted according to weighted differences, and the geometric area of ​​the image objects is used as a constraint, proportionally... Select the top with the highest weighted difference in sequence Each image object is taken as a positive sample, and the image with the lowest weighted difference is selected. Each image object is used as a negative sample.

[0061] Step 4: Network Training and Prediction. Multi-temporal feature vectors are superimposed and imported into a graph convolutional neural network with skip connections. The training samples are used for data training. The classifier predicts all nodes and generates a change result map;

[0062] In this embodiment, GCN learns functions. Extending the concept of convolution from traditional grid-based data to graph data, this function considers nodes Its own feature vector And the feature vectors of neighboring nodes to effectively generate nodes. The representation of graph-based convolutions. A significant difference from grid-based convolutions is that the number of neighbors of a node is not fixed in graph-based convolutions.

[0063] Consider a two-layer GCN for semi-supervised node classification on a graph with a symmetric adjacency matrix A. The forward model takes a simple form:

[0064] (12)

[0065] Consider a multi-layer graph convolutional network (GCN) with the following layer-by-layer propagation rules. The traditional forward propagation model of a GCN neural network is as follows:

[0066] (13)

[0067] in, It is the adjacency matrix of an undirected graph G with added self-connections. To form a weighted adjacency matrix that takes into account the adjacency relationships of adjacent objects, It is the identity matrix. and It is a trainable weight matrix for a specific layer. This represents the activation function. , It is the activation matrix of the l-th layer. .

[0068] During training, as the number of network layers and iterations increases, the hidden layer representations of each node tend to converge to the same value, representing the same location in the network space. This oversmoothing can lead to performance degradation. To better learn neighbor combinations and attributes, a network architecture called Jumping Knowledge Networks was proposed. This architecture adaptively learns to aggregate nodes in different locations into different domains, thereby improving the node representation. Among the three node aggregation methods provided by this architecture, the JK-MaxPool method is adaptive and does not introduce unnecessary parameters. Therefore, choosing this method as the node aggregation method ensures that model efficiency is not reduced while improving model performance.

[0069] After After propagation through the GCN layers, the feature vectors of each node, except for the output layer, are updated as follows: JK-MaxPool considers arbitrary nodes The GCN layer feature aggregation is shown below:

[0070] (14)

[0071] The GCN model with the JK-MaxPool architecture was trained using weighted dissimilarity adaptive samples, and the cross-entropy loss function was used. The model is iteratively optimized using the gradient descent algorithm. After the next iteration, use The classifier performs binary classification on all nodes and generates a change result map. The experimental results are as follows: Figure 2 As shown.

Claims

1. An object-oriented graph neural network unsupervised remote sensing image change detection method, characterized in that, The method comprises the following steps: Step 1: multi-scale and multi-level segmentation and feature selection are performed on two-phase images, a multi-dimensional feature space is established, and a multi-temporal feature vector is obtained; Step 2: a network structure is generated by using the image segmentation vector, a cosine similarity is calculated by using node features, and a first-order weighted adjacency matrix is constructed; Step 3: a weighted difference degree of objects of the two-phase images is calculated, and image objects with the highest and lowest change possibility are adaptively selected as training samples according to a set proportion; Step 4: network training and prediction, the multi-temporal feature vector is superimposed into a graph convolution neural network with a skip connection, data training is performed by using the training sample, and a change result graph is generated by predicting all nodes through the classifier a classifier, and predicting all nodes through the classifier The step 1 comprises: According to the bottom-up principle, the multi-scale segmentation parameters of the group are used respectively The previous layer image is segmented by using the multi-scale segmentation parameters of the group, and a total of The object layer of the remote sensing image object with different sizes has different sizes, and based on the principle of gradually decreasing, the remote sensing image object obtained by using the large segmentation scale parameter value is completely contained in the remote sensing image object obtained by using the small segmentation scale parameter value. The step 2 specifically comprises the following steps: Step 2.1: taking a remote sensing image object as a node, and constructing an adjacent edge between adjacent nodes to generate a first-order adjacency matrix; Step 2.2: the cosine similarity of each node and the adjacent node features is calculated, and a first-order weighted adjacency matrix is constructed, and the formula is as follows: (1) wherein, , is a current feature vector index, and is a feature vector of the th object node and the th object node, the closer the cosine value is to 1, the smaller the difference between the two image objects; the closer the cosine value is to 0, the greater the difference between the two image objects. The step 3 specifically comprises the following steps: Step 3.1: spectral, index, geometric and texture features of the image object are extracted, the difference variance of different feature bands is taken as the weight of feature fusion by using the chi-square transformation, and the multi-level feature weighted difference degree is calculated: (2) (3) in, θ For weight parameters: , These are the multi-scale parameters for object layer 0 and object layer 1, respectively. The number of features of a single-layer image object; For the first Weighted difference of each image, They represent the old and new eras, for 3D feature vectors For the old and new eras The standard deviation of the differential image of each characteristic band; Step 3.2: the weighted difference degree of all remote sensing image objects is taken as a training sample, a change threshold is obtained by using a Bayesian threshold determination method based on an EM algorithm, and the sample is adaptively selected.

2. The object-oriented graph neural network unsupervised change detection method for remote sensing images of claim 1, wherein, The step 1 further comprises: registering and relative radiation correcting the remote sensing images of different time phases; a fractal network evolution algorithm is used for multi-scale segmentation of the old-phase image, the segmentation result is taken as a theme layer for segmentation of the new-phase image, and the segmentation result of the later-phase image is controlled and restricted.

3. The object-oriented graph neural network unsupervised change detection method for remote sensing images of claim 1, wherein, The multi-scale segmentation is based on the principle of maximum homogeneity and minimum heterogeneity of the segmented object to determine the segmentation scale.

4. The object-oriented graph neural network unsupervised change detection method for remote sensing images of claim 1, wherein, The adaptive sample selection in the step 3.2 is implemented in the following process: Setting a change object threshold, all image objects are divided into change objects and non-change objects, respectively sorted according to the weighted difference degree, and the geometric area of the image object is taken as a constraint, and the image objects are sorted according to the weighted difference degree The first The first The first

Images