Change Detection Method Based on Siamese Cascade Difference Neural Network

Abstract

This invention discloses a change detection method based on a Siamese difference neural network. The method comprises the following steps: constructing a symmetrical bilateral cascade neural network structure; establishing a change detection learning model for enhanced feature extraction; optimizing the learning model using an attention mechanism; extracting the optimized results and outputting the changed regions. The invention has the advantage of defining a Siamese change detection model for the change detection problem, effectively mitigating false detections caused by errors in bi-temporal image alignment. The method is highly robust to scale and color variations in images and can adapt to a wide variety of data types.

Description

Technical Field

[0001] The present invention belongs to the field of change detection, and in particular relates to a change detection method based on Siamese cascade difference neural network. Background Art

[0002] Change detection aims to detect changed regions in images of the same scene taken at different times. It is important in many applications, including video surveillance, medical diagnosis and treatment, and particularly in remote sensing monitoring and land use analysis. Bi-temporal CD is a comprehensive image analysis method in which each point in the image is labeled into two groups: changed and unchanged. This analysis yields a final difference map and CD results.

[0003] Traditional change detection methods first perform image preprocessing, such as geometric alignment and radiometric correction; then generate pixel-level differences by comparing features extracted from bi-temporal images; and finally perform image segmentation to classify pixels as changed or unchanged. However, these methods rely heavily on expert experience and have many manually determined parameters, making it difficult to process large amounts of data with high accuracy. Consequently, they present several limitations at the application level. With the rapid development of deep learning methods, numerous neural network models and components have been adopted in the CD field to extract deeper feature representations, making it possible to extract feature maps through end-to-end approaches. While CD methods have achieved practical success, several challenges remain. Many existing methods struggle to accurately capture the spatial locality of an image due to irrelevant inconsistencies such as illumination variations and alignment errors. This can lead to uncertainty in pixel edges and misjudgment of objects.

[0004] Given two images, most co-registration methods treat the captured scene as a flat surface and transform the images using a fixed transformation template, such as shift, rotation, and affine transformation. Therefore, it is difficult to fully align high-resolution images captured from different angles. Such images are common in many scenarios, such as very high-resolution (VHR) optical remote sensing images and those captured by unmanned aerial vehicles (UAVs). Therefore, in many change detection scenarios, methods that are robust to co-registration errors are needed. Object-based change detection methods first classify objects in the images and then compare them, which is robust to co-registration errors. To generate accurate change regions, a dedicated classification method should be designed. An intuitive approach is to classify multi-temporal images separately and then compare the corresponding classes to generate change regions. The accuracy of these methods depends on the accuracy of the classification method, and error propagation is common. Even with high self-alignment and robustness to co-registration errors, object-based methods are often supervised to learn an accurate classifier. Summary of the Invention

[0005] The present invention discloses a change detection method based on Siamese cascade difference neural network, which aims to detect changes in different application scenarios. A change detection network based on bilateral deep structure is proposed to obtain change information through attention mechanism and fusion features.

[0006] The technical solution to achieve the purpose of the present invention is as follows: In a first aspect, the present invention provides a change detection method based on Siamese difference neural network, comprising:

[0007] The first step is to construct a symmetrical Siamese cascade neural network structure for learning. The input at both ends of the network is two registered images, which are passed through several layers of convolutional layers to obtain feature maps at different levels.

[0008] The second step is to define a change detection learning model for enhanced feature extraction and introduce an attention mechanism to optimize the feature map.

[0009] In the third step, the feature maps that have been processed by the attention mechanism are fused and upsampled through jump connections to generate feature maps at different levels. The feature maps are further enhanced by weighting, and the corresponding difference maps are obtained by performing Euclidean distance difference. The difference maps of different sizes are upsampled to the same size, and finally the different difference maps are added together to obtain the final change map.

[0010] In the fourth step, the contrast loss function is introduced as a measure between the change map and the true label map, and the neural network is trained by minimizing the loss.

[0011] In a second aspect, the present application further provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the first aspect when executing the program.

[0012] In a third aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect above.

[0013] In a fourth aspect, the present application further provides a computer program product, comprising a computer program, which implements the method described in the first aspect when executed by a processor.

[0014] Compared with the existing technology, the present invention has the following significant features: (1) a bilateral convolutional neural network architecture is defined; (2) a multi-scale difference attention module and a weighted difference fusion mechanism module are defined to learn network parameters using two input images; (3) the loss function is optimized using batch balanced contrast loss, the purpose of which is to modify the weights of different loss terms in the loss function to achieve a balanced effect.

[0015] This paper uses two input images to learn network parameters in a supervised manner. The neural network consists of two parallel encoders and decoders with shared weights. The former is used to extract multi-scale features, and the latter decodes change information by merging different differential feature maps. Ablation experiments demonstrate the effectiveness of our proposed multi-scale difference attention module and weighted difference fusion mechanism module, which can obtain the correspondence between different features and intermediate semantic change maps at different levels. Both modules have excellent effects in image denoising and compression artifact suppression. Experiments in different data types and scenarios demonstrate the superiority of this method and improve the robustness of co-registration errors.

[0016] The present invention is further described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a flow chart of the present invention.

[0018] Figure 2 This is a content coding network structure diagram proposed by the present invention.

[0019] Figure 3 Detection results of different methods on the dataset. DETAILED DESCRIPTION

[0020] Combine Figure 1 The implementation process of the present invention is described in detail below. The steps are as follows:

[0021] The first step is to build a symmetrical bilateral network structure for feature extraction, such as Figure 2 As shown in the figure, the inputs at both ends of the network are two registered pictures, which are respectively subjected to several layers of convolution to obtain feature maps of different levels.

[0022] The second step is to define the bilateral cascade neural network model. The specific process is:

[0023] (1) Construct a bilateral cascade change detection learning model, define the input data as X, Y, and then define the feature function as follows:

[0024] X i ,Y i =f i (X1,Y1)

[0025] where X i ,Y i Represents two input change detection images, f i (X1, Y1) represents the obtained feature map, i is the number of layers;

[0026] (2) Introducing the attention mechanism to optimize the learning model. The steps of the attention mechanism are as follows:

[0027] ① The input feature map is subjected to three different 1x1 convolutions to obtain three feature vectors, which are used to generate attention weights, represented by Q, K, and V respectively;

[0028] ② Adjust the dimensions of the three vectors Q, K, and V to two-dimensional matrices, respectively using To express;

[0029] ③ General Vector and Multiply the transposed matrix of , and get the weight coefficient A through Softmax;

[0030] ④ Use score normalization, that is, divide by the square root of C′;

[0031]

[0032] Where K is obtained by dimension transformation After matrix transposition, we get Q is obtained by dimension transformation and The number of characteristic channels is the same as that of ; C′ represents the number of characteristic channels;

[0033] ⑤Multiply matrix A by Get the weight vector matrix for each input vector

[0034]

[0035] ⑥ General The vector is reshaped into a three-dimensional feature vector, denoted by Y;

[0036] Y=F(X)

[0037] ⑦After adding, the final output result Z is obtained;

[0038] Z=F(X)+X

[0039] Among them, Z is the output feature map, F(X) is the residual mapping function, which represents the similarity relationship between each feature vector. It is a trainable parameter. The goal of this mechanism is to generate a weight coefficient that is distributed with the data in the entire data space.

[0040] In the third step, the feature maps that have been processed by the attention mechanism are fused and upsampled using skip connections to generate feature maps at different levels. The feature maps are then further enhanced using weighting. Euclidean distance difference is then used to generate the corresponding difference maps. The difference maps of different sizes are then upsampled to the same size and finally summed to generate the final change map.

[0041] First, the feature map after the attention mechanism is upsampled through the jump connection to obtain four pairs of feature maps. At time T1, DX i Indicates that T2 time is DYi i Indicates that each feature map undergoes a 1×1 convolution to perform channel transformation to generate the corresponding weight matrix W i , W i Multiply by the corresponding feature map to get the weighted feature, and the weighted feature map is still DX i and DY i express.

[0042] Then the feature maps at different levels are subjected to Euclidean distance difference processing, and the formula is as follows:

[0043] DI i =E(DX i ,DY i )i=1,2,3,4

[0044] Where E(·) represents the Euclidean distance, DI i It represents the difference map obtained by performing Euclidean distance on different levels. Then the feature maps of different levels are upsampled to the same size. The formula is as follows:

[0045] D i =upsample(DI i )i=1,2,3,4

[0046] Among them D i Represents the four sets of difference maps after upsampling, and then the four sets of difference maps are added together to obtain the final network output change map. The formula is as follows:

[0047] D=D1+D2+D3+D4

[0048] The fourth step is to introduce the contrast loss function as a measure between the change map and the true label map. The neural network is trained by minimizing the loss. The loss function is defined as follows:

[0049]

[0050] Where D represents the prediction result image, M represents the binary label, where M is 1 for changes and 0 for no changes, subscripts b, i, j represent batch size, height, and width respectively; m is the boundary value used to limit the range of the changed pixel pairs, which is set to 2 in the experiment, where P u is the weight of unchanged pixel pair, P c Represents the weights of the change pixel pairs, which are calculated by the label values ​​of the corresponding categories.

[0051] Pu =pos_num

[0052] P c =neg_num

[0053] Among them, pos_num and neg_num represent the number of unchanged pixel pairs and changed pixel pairs, respectively, which are calculated according to the label values ​​of the corresponding categories.

[0054] Unlike previous approaches that encode bi-temporal images separately, this paper designs an encoder-differential attention module to focus on the spatial differences between pixels. To improve the network's generalization, it computes attention weights between any pixels in the bi-temporal images and uses them to generate more discriminative features. To improve feature fusion and avoid vanishing gradients, a multi-scale weighted variance mapping fusion strategy is proposed during the decoding phase, resulting in better change detection results and greater adaptability.

[0055] The effect of the present invention can be further illustrated by the following simulation experiments:

[0056] Simulation conditions

[0057] Simulation Experiment Seasonal Change Detection Dataset: includes 16,000 pairs of seasonal Google Earth images with pixel-by-pixel change detection labels, including 3,000 test samples, 3,000 validation samples, and 10,000 training samples. This dataset can not only provide information on changes in common objects such as buildings and land, but also provide information on changes in many detailed objects such as cars and roads. Here, we use this dataset to verify the feasibility of the proposed probabilistic model in change detection and compare the ability of four unsupervised change detection methods in processing image details. The simulation experiments are all configured under the Windows 11 operating system with an AMD Ryzen 5600X (3.7GHz) and RTX 3060GPU environment, and the program is written in Python and PyCharm 2021.

[0058] The evaluation indicators used in this paper are precision, recall, F1 score, and average intersection-over-union (IOU).

[0059] Simulation content

[0060] This paper uses a seasonal change detection dataset to test the performance of its algorithm. To test the performance of the proposed change detection method based on a Siamese Cascade Difference Neural Network, we compared it with currently popular international change detection algorithms: FC-EF, FC-Siam-diff, FC-Siam-conc, and STANet.

[0061] Analysis of simulation experiment results

[0062] Table 1 shows the comparison results of different evaluation indicators under different change detection algorithms in two data sets. It can be seen from Table 1 that in the S1 data set, the change detection method based on Siamese cascade difference neural network proposed in this invention has strong adaptability, can highlight the changed areas, and avoid the influence of unchanged buildings. Compared with FC-EF, FC-Siam-diff, FC-Siam-conc, and STANet, the accuracy in different evaluation indicators is significantly improved. The results of the method of the present invention and the comparative method on the change detection data set are shown in the figure below. Figure 3 The simulation results of the above real data sets demonstrate the effectiveness of the method of the present invention.

[0063] Table 1 Quantitative evaluation of different algorithms on the dataset

[0064]

[0065] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A change detection method based on Siamese difference neural network, characterized in that The following steps are involved: The first step is to construct a symmetrical Siamese cascade neural network structure for learning. The input at both ends of the network is two registered images, which are passed through several layers of convolutional layers to obtain feature maps at different levels. The second step is to define a change detection learning model for enhanced feature extraction and introduce an attention mechanism to optimize the feature map. The specific steps are as follows: (1) Construct a bilateral cascade change detection learning model, define the input data as X, Y, and then define the feature function as follows: X i ,Y i =f i (X1,Y1) where X i ,Y i Represents two input change detection images, f i (X1, Y1) represents the obtained feature map, i is the number of layers; (2) Introducing the attention mechanism to optimize the learning model. The steps of the attention mechanism are as follows: ① The input feature map is subjected to three different 1x1 convolutions to obtain three feature vectors, which are used to generate attention weights, represented by Q, K, and V respectively; ② Adjust the dimensions of the three vectors Q, K, and V to two-dimensional matrices, respectively using To express; ③ General Vector and Multiply the transposed matrix of , and get the weight coefficient A through Softmax; ④Use score normalization; Where K is obtained by dimension transformation After matrix transposition, we get Q is obtained by dimension transformation and The number of characteristic channels is the same as that of ; C' is used to represent the number of characteristic channels; ⑤Multiply matrix A by Get the weight vector matrix for each input vector ⑥ General The vector is reshaped into a three-dimensional feature vector, denoted by Y; Y=F(X) ⑦After adding, the final output result Z is obtained; Z=F(X)+X Among them, Z is the output feature map, F(X) is the residual mapping function, which represents the similarity relationship between each feature vector; In the third step, the feature maps that have been processed by the attention mechanism are fused and upsampled through jump connections to generate feature maps at different levels. The feature maps are further enhanced by weighting, and the corresponding difference maps are obtained by performing Euclidean distance difference. The difference maps of different sizes are upsampled to the same size, and finally the different difference maps are added together to obtain the final change map. In the fourth step, the contrast loss function is introduced as a measure between the change map and the true label map, and the neural network is trained by minimizing the loss.

2. The change detection method based on Siamese difference neural network according to claim 1, characterized in that In the third step, the feature maps that have passed the attention mechanism are fused and upsampled through jump connections to generate feature maps of different levels. The feature maps are then further enhanced by weighting, and the corresponding difference maps are obtained by performing Euclidean distance difference. The difference maps of different sizes are upsampled to the same size, and finally the different difference maps are added together to obtain the final change map. First, the feature map after the attention mechanism is upsampled through the jump connection to obtain four pairs of feature maps. At time T1, DX i Indicates that T2 is timed with DY i Indicates that each feature map undergoes a 1×1 convolution to perform channel transformation to generate the corresponding weight matrix W i , W i Multiply by the corresponding feature map to get the weighted feature, and the weighted feature map is still DX i and DY i express; Then the feature maps at different levels are subjected to Euclidean distance difference processing, and the formula is as follows: THE i s E(DX i ,DY i )iS1,2,3,4 Where E(·) represents the Euclidean distance, DI i Represents the difference map obtained by performing Euclidean distance on different levels; then the feature maps of different levels are upsampled to the same size, and the formula is as follows: D i =upsample(DI i )i=1,2,3,4 Among them D i Represents the four sets of difference maps after upsampling. Then the four sets of difference maps are added together to obtain the change map of the final network output. The formula is as follows: D=D1+D2+D3+D4.

3. The change detection method based on Siamese difference neural network according to claim 1, characterized in that The fourth step introduces the contrast loss function as a measure between the change map and the true label map. The neural network is trained by minimizing the loss. The loss function is defined as follows: Where D represents the prediction result graph, M represents the binary label, where M is 1 for a change and 0 for no change, and the subscripts b, i, and j represent batch size, height, and width, respectively. m is the boundary value, which is used to limit the range of the changed pixel pairs; P u is the weight of unchanged pixel pair, P c Represents the weights of the change pixel pairs, which are calculated by the label values ​​of the corresponding categories; P u =pos_num P c =neg_num Among them, pos_num and neg_num represent the number of unchanged pixel pairs and changed pixel pairs, respectively, which are calculated according to the label values ​​of the corresponding categories.

4. The change detection method based on Siamese difference neural network according to claim 3, characterized in that m＝2。 5. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the method according to any one of claims 1 to 4 are implemented.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

7. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images