A method for detecting surface terrain changes in planetary remote sensing images based on deep learning

Abstract

The present invention belongs to the technical field of remote sensing image processing and change detection, and particularly relates to a method for detecting surface terrain changes in planetary remote sensing images based on deep learning. The method includes: inputting the dual-temporal planetary remote sensing image data of the area to be detected into a pre-established and trained planetary remote sensing image change model to obtain the detection result of surface terrain changes; the planetary remote sensing image change model extracts multi-layer features of dual-temporal images based on the Siamese network framework, and through a multi-level differential feature fusion structure, comprehensively measures the differential features of different levels to achieve the detection of terrain changes. The planetary remote sensing image change model designed by the present invention not only has high accuracy, but also has the advantages of few parameters and low computational complexity, and has high practical application value.

Description

Technical Field

[0001] The present invention belongs to the technical fields of remote sensing image processing and change detection, and particularly relates to a method for detecting surface terrain changes in planetary remote sensing images based on deep learning. Background Art

[0002] Change detection is to quantitatively analyze and determine the characteristics and processes of surface changes from remote sensing data of different periods. The change detection of remote sensing images essentially is to detect the change information of the surface at different time phases, and is to quantitatively analyze and determine the characteristics and processes of surface changes from remote sensing data of different periods. The Mars probes launched in China will return a large amount of image data during the execution of scientific exploration tasks. It is of great significance to analyze the data collected by the planetary exploration mission using change detection algorithms, mainly for the mining and extraction of valuable information in the time dimension. At present, the application scenarios of change detection in the process of planetary exploration mainly include observing the changes in planetary landforms caused by meteorite impacts and observing the slope line terrain changes on the surface of Mars, which is of great significance for obtaining dynamic change information of key areas of concern and timely adjusting response plans. In addition, when the satellite is in the staring observation state, that is, when the camera viewing angle attitude does not change and continuous photographs are taken of a certain target area at the same position, by performing on-orbit change detection on the sequence of images, it is also possible to remove duplicate images, extract effective information, and reduce the amount of data to be downlinked to a greater extent, alleviating the bandwidth pressure.

[0003] With the popularization of high-resolution remote sensing images, change detection has become one of the most studied topics in the field of remote sensing. Traditional image change detection methods include steps such as image processing, difference map generation, and difference map analysis. The mainstream methods in the difference map generation stage include the difference method, ratio method, wavelet fusion, logarithmic ratio method, and some related improved operators. Most of these methods require domain-specific prior knowledge, and then manually design and extract features such as the texture, morphology, and neighborhood of the image for change detection. The extracted image features are relatively shallow, making it difficult to completely model the image, and it is time-consuming and laborious. In recent years, change detection methods based on deep learning have emerged and gradually become the preferred tool for remote sensing image analysis. The application of deep learning in remote sensing image change detection has an ideal effect on solving the limitations of image processing algorithms, improving the accuracy of change detection, and the degree of automation of the process.

[0004] Based on deep learning methods, designing a more intelligent change detection process for planetary remote sensing images by building a network and defining optimization rules has important research value and theoretical significance for further improving the accuracy of change detection and better exerting the potential application value of deep learning models.

[0005] In recent years, many organizations and scholars at home and abroad have explored the change detection technology for planetary remote sensing images. Most of them adopt solutions that divide multiple sub-problems and solve them separately, such as feature extraction + post-classification or difference map generation + analysis and decision-making. There are still disadvantages such as cumbersome steps and error accumulation. Summary of the Invention

[0006] The purpose of the present invention is to overcome the defects of the prior art. The present invention proposes a new change detection model and method for planetary remote sensing images based on deep learning to obtain a faster execution speed and better detection accuracy results. The new change detection algorithm is a solution that can directly obtain the desired change detection output result from the original dual-temporal image data. The entire change detection model mainly includes a dual-temporal image feature extraction module, a multi-level feature difference fusion module, and a classification decision module. During the training process of the change detection model, no artificial sub-problem division is performed, and an end-to-end learning method is adopted, which can more likely obtain the global optimal solution.

[0007] To achieve the above purpose, the present invention proposes a method for detecting surface terrain changes in planetary remote sensing images based on deep learning. The method includes:

[0008] Input the dual-temporal planetary remote sensing image data of the area to be detected into the pre-established and trained planetary remote sensing image change model to obtain the surface terrain change detection result;

[0009] The planetary remote sensing image change model extracts multi-layer features of dual-temporal images based on the Siamese network framework, and comprehensively measures the difference features at different levels through a multi-level difference feature fusion structure to achieve the detection of terrain changes.

[0010] As an improvement of the above method, the input of the planetary remote sensing image change model is the remote sensing image data registered in the same area to be detected on the planet at time T1 and time T2. The types of the remote sensing image data include visible light images and multi-spectral images, and the output is the detection result of whether the surface terrain has changed. The planetary remote sensing image change model includes a dual-temporal image feature extraction module, a multi-level feature difference fusion module, and a classification decision module. Among them,

[0011] The dual-temporal image feature extraction module is used to extract multi-layer features of dual-temporal images and enhance the visual feature representation ability of local and global areas;

[0012] The multi-level feature difference fusion module is used to comprehensively measure the difference features at different levels and enhance the feature expression ability;

[0013] The classification decision module is used to output the detection result, where 0 indicates no change and 1 indicates a change.

[0014] As an improvement of the above method, the dual-temporal image feature extraction module adopts a basic Siamese network framework, including two improved MobileViT networks with the same structure. The improved MobileViT network uses the MobileViT backbone network part as a feature extractor, removes the original MobileViT's final pooling layer and classifier, and sets three feature positions at the output of the mvit1 module, the output of the mvit2 module, and the output of the improved MobileViT network in each improved MobileViT network, so as to obtain the first-level feature Stage1_T1, the second-level feature Stage2_T1, and the third-level feature Stage3_T1 of the planetary remote sensing image data to be detected at time T1, as well as the first-level feature Stage1_T2, the second-level feature Stage2_T2, and the third-level feature Stage3_T2 of the planetary remote sensing image data to be detected at time T2.

[0015] As an improvement of the above method, the processing process of the multi-level feature difference fusion module MFDF includes:

[0016] Calculate the absolute differences between the features Stage1_T1 and Stage1_T2, Stage2_T1 and Stage2_T2, and Stage3_T1 and Stage3_T2 respectively to obtain the corresponding first-level absolute difference Diff1, second-level absolute difference Diff2, and third-level absolute difference Diff3;

[0017]

[0018] Perform a concatenation operation on Diff1 and Diff2 to obtain the first concatenated output Fdf1, then sequentially input Fdf1 into a 3×3 standard convolutional layer and a max pooling layer to obtain the second concatenated output Fdf2, and then concatenate Fdf2 with Diff3 to obtain the third concatenated output Fdf3, and output it to the classification decision module.

[0019] As an improvement of the above method, the classification decision module includes an average pooling layer and a fully connected layer.

[0020] As an improvement of the above method, the method further includes a training step for the planetary remote sensing image change model; specifically including:

[0021] Establish a training set using publicly available Mars image data;

[0022] Batch input the sample data in the training set into the planetary remote sensing image change model, calculate the cross-entropy loss function, perform gradient backpropagation, use the Adam optimizer to update the network weights, and iteratively update until the training requirements are met to obtain a trained planetary remote sensing image change model.

[0023] On the other hand, the present invention proposes a system for detecting surface terrain changes in planetary remote sensing images based on deep learning, and the system includes: a detection output module and a planetary remote sensing image change model; wherein,

[0024] The detection output module is used to input the dual-temporal remote sensing image data of the planetary area to be detected obtained into the pre-established and trained planetary remote sensing image change model to obtain the detection result of surface terrain changes;

[0025] The planetary remote sensing image change model extracts multi-layer features of dual-temporal images based on the twin network framework, and through a multi-level differential feature fusion structure, comprehensively measures the differential features at different levels to achieve the detection of terrain changes.

[0026] Compared with the prior art, the advantages of the present invention are as follows:

[0027] 1. The present invention innovatively designs a planetary remote sensing image change detection model based on the twin network framework, in which the backbone network uses the MobileViT model to extract multi-layer features of dual-temporal images, which can effectively combine the inductive bias advantages of CNN and the global information modeling advantages of vision transformers, and enhance the local and global visual feature representation capabilities;

[0028] 2. The present invention designs a multi-level differential feature fusion structure MDCF, which comprehensively measures the differential features at different levels and enhances the discriminant ability of the model for the differential change information of dual-temporal images;

[0029] 3. The change detection model designed by the present invention not only has high accuracy, but also has the advantages of few parameters and low computational complexity, and has high practical application value. Description of the Drawings

[0030] Figure 1 is the model training and prediction flow chart of the present invention;

[0031] Figure 2 is the structural block diagram of the planetary remote sensing image change model of the present invention;

[0032] Figure 3 is the overall implementation step diagram of the method for detecting surface terrain changes in planetary remote sensing images based on deep learning of the present invention. Detailed Embodiments

[0033] The image data targeted by this algorithm is planetary remote sensing image data obtained by observing the same area at different times, mainly for the registered dual-temporal images, and the data type can be visible light images or multi-spectral images, etc.

[0034] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and embodiments.

[0035] Embodiment 1

[0036] Aiming at the problems of cumbersome calculation steps and high complexity in the existing related research methods, Embodiment 1 of the present invention aims at change detection at the scene level within the planetary surface range, and designs a method for change detection of planetary remote sensing images based on deep learning. The overall technical flow chart is as Figure 1 shown, which mainly includes the following three steps:

[0037] Step 1: First is the construction of the neural network model. The structure of the planetary remote sensing image change model TransPCD designed by the present invention is as Figure 2 shown. This change detection model mainly includes the following components:

[0038] 1) Dual-temporal image feature extraction module: This model adopts a basic Siamese network framework, and uses two MobileViT network structures with shared parameters to perform multi-level feature extraction of dual-temporal images, and respectively selects and obtains the feature levels of the dual-temporal images Stage1_T1, Stage2_T1, Stage3_T1 and Stage1_T2, Stage2_T2 and Stage3_T2. The MobileViT network combines the advantages of convolutional neural networks and the transformer structure based on self-attention mechanism. The mvit module in it uses transformer operations to replace the local processing steps in standard convolutional operations, overcomes the disadvantage of limited receptive fields of convolutional layers, and can effectively represent the learned local and global visual information with fewer parameters. Here, the backbone network part of MobileViT is used, and the original network's final pooling layer and classifier are removed.

[0039] 2) Multi-level feature difference fusion structure (MFDF): Comprehensively measure the difference features at different levels to enhance the feature expression ability of the model. The calculation process is as follows:

[0040] ① Input Stage1_T1~Stage3_T1 and Stage1_T2~Stage3_T2 into the MFDF module;

[0041] ② Calculate the absolute difference of each level of features:

[0042]

[0043] ② Then, cascade the two - level differential features Diff1 and Diff2 to obtain Fdf1. Then, input Fdf1 into a standard 3×3 convolutional layer and a max - pooling layer in sequence to obtain Fdf2. Then, cascade the output with Diff3 to obtain Fdf3, which is the output of the MFDF module;

[0044] 3) Classification decision module: Input Fdf3 into an average - pooling layer and a fully - connected layer in sequence to obtain the final output result.

[0045] Step 2: After the model is constructed, use the existing training set and validation set to train and evaluate the model respectively. During the training process, the cross - entropy loss function is used as the loss function, and the Adam optimizer is selected as the optimizer. In the training stage, use the training set to train the model. Input the sample data into the model batch - by - batch, calculate the cross - entropy loss function, perform gradient backpropagation, and use the Adam optimizer to update the network weights. Iteratively update until the accuracy on the validation set no longer improves, and then end the training;

[0046] Step 3: After the training is completed, in order to detect the effect of the TransPCD model designed in the present invention, we analyze the model test results on the test data set and conduct a comparative experiment with the existing change - detection methods. The accuracy evaluation index used is the detection accuracy rate.

[0047] As Figure 3 shown, the implementation process of the entire planetary remote - sensing image terrain and surface change - detection algorithm includes the following steps:

[0048] (1) First, obtain the dual - temporal planetary remote - sensing image data;

[0049] (2) Input the dual - temporal images to be measured into the trained model to obtain the prediction results;

[0050] (3) After obtaining the prediction results of the neural - network model, it can provide a strong basis for subsequent in - depth research and verification by planetary scientists.

[0051] The experimental results on the publicly available Mars image data verify the effectiveness of the algorithm in this paper. The experimental results show that the average detection accuracy of the model proposed in this paper reaches 96.5%, the accuracy rate is higher than that of related algorithms, and the model size is 1.38MB, and the computational cost is 0.19GFlops. The comprehensive performance is better, and efficient planetary remote - sensing image change detection is achieved.

[0052] Example 2

[0053] Example 2 of the present invention proposes a system for detecting surface terrain changes in planetary remote - sensing images based on deep learning, which is implemented by the method of Example 1. The system includes: a detection output module and a planetary remote - sensing image change model; among them,

[0054] A detection output module, configured to input the acquired dual-temporal remote sensing image data of the planet to-be-detected area into a pre-established and trained planet remote sensing image change model to obtain a surface terrain change detection result;

[0055] The planet remote sensing image change model extracts multi-layer features of dual-temporal images based on a siamese network framework, and through a multi-level differential feature fusion structure, comprehensively measures differential features at different levels to achieve terrain change detection.

[0056] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the embodiments, those of ordinary skill in the art should understand that any modification or equivalent replacement of the technical solutions of the present invention does not depart from the spirit and scope of the technical solutions of the present invention, and they should all be covered within the scope of the claims of the present invention.

Claims

1. A method for detecting surface terrain changes in planetary remote sensing images based on deep learning, the method comprising: Inputting the dual-temporal planetary remote sensing image data of the area to be detected into a pre-established and trained planetary remote sensing image change model to obtain the detection result of surface terrain changes; The planetary remote sensing image change model extracts multi-layer features of dual-temporal images based on the Siamese network framework, and through a multi-level difference feature fusion structure, comprehensively measures the difference features at different levels to achieve the detection of terrain changes; The input of the planetary remote sensing image change model is the registered remote sensing image data of the same area to be detected on the planet at time T1 and time T2. The types of the remote sensing image data include visible light images and multi-spectral images, and the output is the detection result of whether the surface terrain has changed. The planetary remote sensing image change model includes a dual-temporal image feature extraction module, a multi-level feature difference fusion module, and a classification decision module; wherein, The dual-temporal image feature extraction module is used to extract multi-layer features of dual-temporal images and enhance the visual feature representation ability of local and global areas; The multi-level feature difference fusion module is used to comprehensively measure the difference features at different levels and enhance the feature expression ability; The classification decision module is used to output the detection result, where 0 indicates no change and 1 indicates a change; The dual-temporal image feature extraction module adopts a basic Siamese network framework, including two improved MobileViT networks with the same structure. The improved MobileViT network uses the MobileViT backbone network part as a feature extractor, removes the original MobileViT's final pooling layer and classifier, and sets three feature positions at the output of the mvit1 module, the output of the mvit2 module, and the output of the improved MobileViT network in each improved MobileViT network respectively, so as to obtain the first-level feature Stage1_T1, the second-level feature Stage2_T1, and the third-level feature Stage3_T1 of the planetary remote sensing image data to be detected at time T1, and the first-level feature Stage1_T2, the second-level feature Stage2_T2, and the third-level feature Stage3_T2 of the planetary remote sensing image data to be detected at time T2.

2. The method for detecting surface terrain changes in planetary remote sensing images based on deep learning according to claim 1, wherein The processing process of the multi-level feature difference fusion module MFDF includes: Calculating the absolute differences between the features Stage1_T1 and Stage1_T2, Stage2_T1 and Stage2_T2, and Stage3_T1 and Stage3_T2 respectively to obtain the corresponding first-level absolute difference Diff1, second-level absolute difference Diff2, and third-level absolute difference Diff3; Performing a concatenation operation on Diff1 and Diff2 to obtain the first concatenated output Fdf1, then sequentially inputting Fdf1 into a 3×3 standard convolutional layer and a max pooling layer to obtain the second concatenated output Fdf2, and then concatenating Fdf2 and Diff3 to obtain the third concatenated output Fdf3, and outputting it to the classification decision module.

3. The method for detecting surface terrain changes in planetary remote sensing images based on deep learning according to claim 1, characterized in that, The classification decision module includes an average pooling layer and a fully connected layer.

4. The method for detecting surface terrain changes in planetary remote sensing images based on deep learning according to claim 1, characterized in that The method further includes a training step of a planetary remote sensing image change model; specifically including: Using publicly available Mars image data to establish a training set; Inputting the sample data in the training set into the planetary remote sensing image change model in batches, calculating the cross-entropy loss function, performing gradient backpropagation, using the Adam optimizer to update the network weights, and iteratively updating until the training requirements are met to obtain a trained planetary remote sensing image change model.

5. A system for the method of detecting surface terrain changes in planetary remote sensing images based on deep learning according to claim 1, characterized in that, The system includes: a detection output module and a planetary remote sensing image change model; wherein, The detection output module is used to input the dual-temporal remote sensing image data of the planetary area to be detected into the pre-established and trained planetary remote sensing image change model to obtain the detection result of surface terrain change; The planetary remote sensing image change model extracts multi-layer features of dual-temporal images based on the Siamese network framework, and comprehensively measures the differential features of different levels through a multi-level differential feature fusion structure to achieve terrain change detection.

Images