An unsupervised-based planetary change detection system, method, device, and medium

Abstract

This invention discloses an unsupervised method for detecting planetary changes, belonging to the field of remote sensing image processing technology, and includes the following steps: S1, acquiring two planetary remote sensing images at different time phases, namely, planetary surface image X at time T1. i Planetary surface image at time T2 Y i The images are preprocessed to obtain preprocessed images. A pre-trained VGG-19 network and a Convolutional Block Attention (CBAM) module are used to extract deep features from the preprocessed images. Global max pooling is then used to obtain patch-level features of the two-phase planetary images. Further analysis of the differences in deep features between the two-phase images is then performed to determine whether changes have occurred. The planetary change detection method provided by this invention not only has high accuracy but also requires few parameters and has low computational complexity. It has high practical application value on different planetary datasets and can be further studied using unsupervised learning on different datasets in the future.

Description

Technical Field

[0001] This invention belongs to the field of remote sensing image processing technology, specifically relating to an unsupervised planetary change detection system, method, device, and medium. Background Technology

[0002] Change detection (CD) is used to detect changes in the Earth's surface across different time phases and is an important application of remote sensing technology. In recent years, to enrich our understanding of the solar system, planetary exploration missions have gradually increased, and scientific imaging instruments for various purposes have played a crucial role, such as the High Resolution Imaging Science Experiment (HiRISE) and the Background Imager (CTX) for the Mars Orbiter. CD plays a key role in many environmental missions, such as disaster management, urban monitoring, and military applications. While CD plays a significant role in Earth exploration, it has not been explored as extensively in the context of planetary exploration. Furthermore, using change detection algorithms to study planetary surface changes caused by impact craters and planetary evolution is of great significance.

[0003] Pixel-based change detection models are the earliest change detection methods and have been extensively studied and widely applied. Kerner et al. investigated several supervised methods against a planetary CD background and applied them to monitor changes caused by meteorite impacts and to monitor recurring slope lines (RSLs) that appear / disappear on the Martian surface on a timescale of nearly one year. However, the accuracy of the model trained on the HiRISERSL dataset decreased significantly when applied to the CTX meteorite impact dataset. Sudipan et al. proposed an unsupervised planetary change detection method based on this, using the sixth convolutional layer of VGG-16 as the pre-trained model, but the model accuracy and parameter count need further improvement. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art and provide an unsupervised planetary change detection system, method, device and medium.

[0005] The first objective of this invention is to provide an unsupervised method for detecting planetary changes, comprising the following steps:

[0006] S1. Acquire two remote sensing images of the planet at different time points, namely, the planetary surface image X at time T1. i (i = 1, 2, ..., n) and planetary surface image Y at time T2 i (i = 1, 2, ..., n), and perform preprocessing to obtain preprocessed images respectively;

[0007] S2. Perform feature extraction on the two preprocessed images obtained in step S1 to obtain X. i and Y iMulti-temporal depth feature maps;

[0008] S3, regarding the X obtained in step S2 i and Y i The multi-temporal deep feature maps are enhanced separately to obtain X. i and Y i Feature enhancement map;

[0009] S4. Perform global max pooling on the feature layer vectors in the two feature enhancement maps obtained in step S3, and output the max pooled features to obtain X. i and Y i The registered images;

[0010] S5. Perform feature fusion on the two registered images obtained in step S4 to obtain the K dimension of D. Xi and K Yi ;

[0011] S6. The K obtained in step S5 Xi and K Yi By comparison, the change index M is obtained. i By M i Compared with a threshold τ, if Mi is less than τ, it indicates that the planetary remote sensing image X is from a different time phase. i and Y i No change; otherwise, it represents a planetary remote sensing image X from a different time phase. i and Y i Things have changed.

[0012] Preferably, step S2 specifically includes the following steps:

[0013] S21. Pass the two preprocessed images obtained in step S1 as inputs to the VGG-19 model;

[0014] The 16 convolutional layers of the S22 and VGG-19 models sequentially extract features from the two preprocessed images to obtain X. i and Y i The multi-temporal deep feature map has a dimension of R×C×D, where D is the number of features.

[0015] Preferably, step S3 specifically includes the following steps:

[0016] X obtained in step S2 i and Y i The multi-temporal depth feature maps are input into the CBAM feature enhancement module to obtain X sequentially. i and Y i The feature map and feature enhancement map are obtained, and the obtained feature enhancement map is used as the output of the CBAM feature enhancement module to obtain X.i and Y i Feature enhancement map.

[0017] Preferably, the CBAM feature enhancement module is composed of a channel attention module and a spatial attention module connected in series. The channel attention module is used to obtain a feature map containing more key channel information, and the spatial attention module is used to obtain a feature enhancement map containing more key spatial location information.

[0018] A second objective of this invention is to provide an unsupervised planetary change detection system, comprising:

[0019] The data preprocessing module is used to acquire two planetary remote sensing images from different time periods and perform preprocessing on them to obtain preprocessed images.

[0020] The feature extraction module is used to extract features from the preprocessed image obtained by the data preprocessing module to obtain multi-temporal depth feature maps.

[0021] The CBAM feature enhancement module is used to enhance the multi-temporal deep feature maps obtained by the feature extraction module, resulting in an enhanced feature map.

[0022] The registration module performs global max pooling on the feature enhancement map obtained by the feature enhancement module to obtain the registered image.

[0023] The feature fusion module is used to perform feature fusion on the registered image obtained by the registration module to obtain the K dimension of D. Xi and K Yi ;

[0024] The deep feature difference comparison module is used to compare the K values ​​obtained from the feature aggregation module. Xi and K Yi By comparison, the change index M is obtained. i By M i By comparing with a threshold τ, the planetary remote sensing images X at different time phases are determined. i and Y i Has anything changed?

[0025] Preferably, the CBAM feature enhancement module is composed of a channel attention module and a spatial attention module connected in series. The channel attention module is used to obtain a feature map containing more key channel information, and the spatial attention module is used to obtain a feature enhancement map containing more key spatial location information.

[0026] A third objective of this invention is to provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps corresponding to the above-described planetary change detection method.

[0027] A fourth objective of this invention is to provide a computer storage medium storing computer program instructions, characterized in that the program instructions, when executed by a processor, are used to implement the steps corresponding to the aforementioned planetary change detection method.

[0028] Compared with the prior art, the advantages of this invention are as follows:

[0029] (1) This invention proposes an unsupervised planet detection method, which uses a pre-trained VGG-19 network and a convolutional block attention module (CBAM) to extract deep features. Through global max pooling, patch-level features of dual-temporal planet images are obtained. The differences in deep features of dual-temporal images are further analyzed to determine whether there has been a change. Therefore, it can process various unlabeled multi-temporal planet data without using any labels.

[0030] (2) The planetary change detection method provided by this invention is not only highly accurate, but also has few parameters and low computational complexity. It has high practical application value on different planetary datasets and can be used for unsupervised learning on different data in the future. Attached Figure Description

[0031] Figure 1 A flowchart illustrating an unsupervised planetary change detection method provided in an embodiment of the present invention;

[0032] Figure 2 This is a flowchart of the VGG-19 model extracting features from a preprocessed image in an embodiment of the present invention;

[0033] Figure 3 This is a flowchart illustrating the feature enhancement process of the CBAM feature enhancement module in this embodiment of the invention.

[0034] Figure 4 Images of the lunar surface at time T1 and time T2, and a change feature map obtained using the detection method of this invention;

[0035] In the picture, Figure 4 (a) is an image of the lunar surface at time T1, data from NAC M181330922L; Figure 4 (b) is an image of the lunar surface at time T2, data from NAC M1139065512L; Figure 4 (c) is a change feature map obtained by the detection method of the present invention. Detailed Implementation

[0036] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0037] Example 1

[0038] To address the issues of cumbersome and complex computational steps in existing research methods, this invention aims to detect scene-level changes on the planetary surface and designs an unsupervised planetary change detection method. The overall technical flowchart is shown below. Figure 1 As shown, the specific steps include:

[0039] S1. Acquire two remote sensing images of the planet at different time points, namely, the planetary surface image X at time T1. i (i = 1, 2, ..., n) and planetary surface image Y at time T2 i (i = 1, 2, ..., n), and preprocessed to obtain preprocessed images. These images meet the resolution requirements and contain clear planetary surface texture features. X i and Y i Images of the same region of a planet may not be registered. Preprocessing includes: image projection transformation, radiometric correction, cropping each pair of images to 100×100 pixels, and generating corresponding labels to obtain the preprocessed images, which will be used as the test dataset.

[0040] S2. Perform feature extraction on the two preprocessed images obtained in step S1 to obtain X. i and Y i The multi-temporal deep feature map specifically includes the following steps:

[0041] S21. The two preprocessed images obtained in step S1 are respectively passed as inputs to the preprocessed VGG-19 model.

[0042] The 16 convolutional layers of the S22 and VGG-19 models sequentially extract features from the two preprocessed images to obtain X. i and Y i The multi-temporal deep feature map has a dimension of R×C×D, where D is the number of features. The specific processing procedure is as follows: Figure 2 As shown;

[0043] S3, regarding the X obtained in step S2 i and Y i The multi-temporal deep feature maps are enhanced separately to obtain X. i and Yi The feature enhancement map specifically includes the following steps:

[0044] X obtained in step S2 i and Y i The multi-temporal depth feature maps are input into the CBAM feature enhancement module to obtain X sequentially. i and Y i The feature map and feature enhancement map are obtained, and the obtained feature enhancement map is used as the output of the CBAM feature enhancement module to obtain X. i and Y i Feature enhancement map.

[0045] The CBAM feature enhancement module consists of a channel attention module and a spatial attention module connected in series, with the architecture as follows: Figure 3 As shown. The channel attention module is used to obtain a feature map containing more key channel information, and the spatial attention module is used to obtain a feature enhancement map containing more key spatial location information, specifically:

[0046] The network first learns which features are key features through the channel attention module, and then uses the spatial attention module to learn where the key features are located, thereby enhancing the acquisition of image discriminative features. In the CBAM process, the input features of CBAM are denoted by F, and the output features of the channel attention mechanism and the final features output by the spatial attention mechanism are denoted by F1 and F2, respectively. C and F R express:

[0047]

[0048]

[0049] Among them, M C M represents the refined features obtained by multiplying the input features by the channel attention map. S Represents the sequence space attention map. This indicates that element-based multiplication allows the input feature map F to be processed by the channel attention module, resulting in a feature map F containing more critical channel information. C Then, the FC is processed through a spatial attention module to obtain a feature-enhanced map F containing more key spatial location information. R This is then used as the final output feature map of CBAM.

[0050] S4. Since the feature enhancement map obtained in step S3 is very sensitive to the position of the input image, and errors may occur due to potential image misregistration, global max pooling is performed on the feature layer vectors of the two feature enhancement maps obtained in step S3 respectively. This makes the image feature position changes robust, and the pooled max pooled features are output to obtain X. i and Y iThe registered images;

[0051] S5. Perform feature fusion on the two registered images obtained in step S4. Specifically, accumulate the global pooling feature vectors in alignment, and output the accumulated fused pooling feature as an enhanced robust feature for local information. This overcomes the defect of frequent occurrence of irrelevant features in average pooling, and enhances the feature representation capability of the model. Global pooling will... i and Y i The obtained R×C×D feature maps are respectively converted into D-dimensional K Xi and K Yi ;

[0052] S6. The K obtained in step S5 Xi and K Yi Comparison, specifically K Xi Subtract K Yi The change index M is obtained by taking the norm of the difference. i h represents the process:

[0053] M i =h(K Xi ,K Yi )

[0054] M i The larger the value, the stronger X is. i and Y i It is easier to change, by M i By comparing with a threshold τ, the planetary remote sensing image X at different time phases can be determined. i and Y i Has a change occurred? If Mi is less than τ, it indicates that the planetary remote sensing image X is from a different time period. i and Y i No change; otherwise, it represents a planetary remote sensing image X from a different time phase. i and Y i Things have changed.

[0055] The threshold τ mentioned above is determined by the following method:

[0056] Choose any X from X. i By adding Gaussian noise and random pixel shifting, we obtain X' i , X' i The calculation formula is as follows:

[0057] X i '=g(X i )

[0058] Where g(.) is equivalent to applying Gaussian noise and shifting the image slice by a few pixels, the embodiment of this invention applies a displacement of up to 8 pixels, due to X i and X'i In the same scene, they can be considered pseudo-invariant pairs, but they usually differ slightly from those observed in multi-temporal planetary images. i and X' i Following step S6, we obtain M' i M' i for:

[0059] M i '=h(X i ,X i ')

[0060] M' i This is the sample M-value of the invariant pairs. Similarly, for i = 1, ..., n, it provides the M-distribution of invariant pairs for this invention. The maximum M-value of the invariant pairs can be used as a threshold τ to distinguish between invariant and changing time-relative pairs. τ is expressed as:

[0061] τ=max(M1',M2',…,M i ')

[0062] Considering the effects of random errors, M' is excluded when calculating τ. i The extreme values.

[0063] This invention also provides an unsupervised planetary change detection system, comprising:

[0064] The data preprocessing module is used to acquire two planetary remote sensing images from different time periods and perform preprocessing on them to obtain preprocessed images.

[0065] The feature extraction module is used to extract features from the preprocessed image obtained by the data preprocessing module to obtain multi-temporal depth feature maps.

[0066] The CBAM feature enhancement module is used to enhance the multi-temporal deep feature maps obtained by the feature extraction module, resulting in an enhanced feature map.

[0067] The registration module performs global max pooling on the feature enhancement map obtained by the feature enhancement module to obtain the registered image.

[0068] The feature fusion module is used to perform feature fusion on the registered image obtained by the registration module to obtain the K dimension of D. Xi and K Yi ;

[0069] The deep feature difference comparison module is used to compare the K values ​​obtained from the feature aggregation module. Xi and K Yi By comparison, the change index M is obtained. i By M i By comparing with a threshold τ, the planetary remote sensing images X at different time phases are determined.i and Y i Has anything changed?

[0070] Preferably, the CBAM feature enhancement module is composed of a channel attention module and a spatial attention module connected in series. The channel attention module is used to obtain a feature map containing more key channel information, and the spatial attention module is used to obtain a feature enhancement map containing more key spatial location information.

[0071] This invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps corresponding to the above-described planetary change detection method.

[0072] This invention also provides a computer storage medium storing computer program instructions, characterized in that the program instructions, when executed by a processor, are used to implement the steps corresponding to the above-described planetary change detection method.

[0073] Comparative Example 1

[0074] The detection method provided in this comparative example is the same as the detection method provided in Example 1, except that this comparative example does not include the processing steps S2 and S4.

[0075] Comparative Example 2

[0076] The detection method provided in this comparative example is the same as the detection method provided in Example 1. The only difference is that this comparative example does not include the processing step S4, and the model in step S2 is a k-Means clustering module.

[0077] Comparative Example 3

[0078] The detection method provided in this comparative example is the same as the detection method provided in Example 1, the only difference being that the model in step S2 of this comparative example is the VGG-16 model.

[0079] The detection method provided in the embodiments of the present invention will be verified below.

[0080] The detection method provided in this embodiment of the invention is verified based on the Lunar Reconnaissance Orbiter Narrow Angle Camera (LRO NAC) image dataset and the Mars Exploration Orbiter Background Camera (MRO CTX) image dataset. Mi obtained in step 6 is compared with the threshold τ obtained in step 7. If Mi is less than τ, it indicates no change; otherwise, it indicates a change.

[0081] (1) Taking the LRO NAC dataset as an example

[0082] like Figure 4 (a) and Figure 4As shown in (b), although there is a large misalignment error between the surface images at time T1 and time T2, through... Figure 4 (c) It can be seen that the detection method provided by the embodiments of the present invention can still distinguish between changing patches and unchanged patches, indicating that the feature extraction module and the CBAM feature enhancement module can be used together to accurately extract dual temporal feature content, and can accurately identify changing features by cooperating with the registration module for global pooling.

[0083] (2) Taking the MRO CTX impact crater dataset as an example

[0084] For the MRO CTX impact crater dataset, the transfer learning capabilities of the unsupervised detection method proposed in this embodiment of the invention are compared with those of other supervised methods. The comparison results are shown in Table 1 below:

[0085]

[0086] As shown in Table 1, the unsupervised detection method proposed in this invention outperforms supervised methods (Inception-v3 and SVM) in accuracy. Furthermore, the proposed method outperforms algorithms based on average pooling, k-means clustering, global max pooling + VGG16, and the unsupervised RCVA algorithm, achieving an accuracy of 87.4%. Regarding model parameters, the method provided in this invention reduces the number of parameters by using global max pooling, and the VGG19 module further enhances detection efficiency.

[0087] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. An unsupervised-based planetary change detection method, characterized in that, The method comprises the following steps: S1, acquiring two images of a planet at different time, respectively, the planet surface image X at T1 time i (i=1, 2, …, n) and the planet surface image Y at T2 time i (i=1, 2, …, n), and pre-processing, respectively, to obtain pre-processed images; S2, respectively performing feature extraction on the two preprocessed images obtained in step S1 to obtain X i and Y i multi-time depth feature maps, specifically comprising the following steps: S21, the two preprocessed images obtained in step S1 are respectively input into a VGG-19 model; S22, the 16 convolutional layers of the VGG-19 model respectively sequentially extract features from the two preprocessed images to obtain X i and Y i multi-time depth feature maps, the dimension of the feature map is RxCxD, and D is the number of features. S3, performing feature enhancement on the multi-time depth feature maps of X i and Y i respectively to obtain feature enhanced maps of X i and Y i respectively; S4, performing global max-pooling on the feature layer vectors in the two feature enhanced maps obtained in step S3 respectively, and outputting the maximum pooling features after pooling to obtain X i and Y i registered images; S5, respectively performing feature fusion on the two registered images obtained in step S4 to obtain K Xi and K Yi ; S6、comparing K Xi and K Yi to obtain a change indicator M i , by comparing M i with a threshold τ, if M i is less than τ, it indicates that the planetary remote sensing images X i and Y i of different time phases do not change, otherwise, it indicates that the planetary remote sensing images X i and Y i of different time phases change.

2. The unsupervised-based planetary change detection method of claim 1, wherein, Step S3 specifically comprises the following steps: The multi-time depth feature maps of X i and Y i obtained in step S2 are respectively input into a CBAM feature enhancement module, and the feature maps and feature enhancement maps of X i and Y i are obtained in sequence. The obtained feature enhancement maps are taken as the output of the CBAM feature enhancement module, and the feature enhancement maps of X i and Y i are obtained.

3. The unsupervised-based planetary change detection method of claim 2, wherein, The CBAM feature enhancement module is composed of a channel attention module and a spatial attention module in series, the channel attention module is used to obtain a feature map containing more key channel information, and the spatial attention module is used to obtain a feature enhancement map containing more spatial position key information.

4. An unsupervised-based planetary change detection system, characterized in that, It comprises: A data preprocessing module is configured to acquire two planetary remote sensing images of different time phases, and pre-process the images respectively to obtain pre-processed images; A feature extraction module is configured to perform feature extraction on the pre-processed images obtained by the data preprocessing module to obtain multi-time depth feature maps; A CBAM feature enhancement module is configured to perform feature enhancement on the multi-time depth feature maps obtained by the feature extraction module to obtain feature enhancement maps; A registration module is configured to perform a global maximum pooling operation on the feature enhancement maps obtained by the feature enhancement module to obtain a registration image. The feature fusion module is configured to perform feature fusion on the registered images obtained by the registration module to obtain K Xi and K Yi ; a deep feature difference value comparison module, configured to compare the K Xi and K Yi obtained by the feature aggregation module to obtain a change index M i , and determine whether the planetary remote sensing images X i and Y i of different time phases change by comparing M i with a threshold τ.

5. The unsupervised-based planetary change detection system of claim 4, wherein, The CBAM feature enhancement module is composed of a channel attention module and a spatial attention module in series, the channel attention module is used to obtain a feature map containing more key channel information, and the spatial attention module is used to obtain a feature enhancement map containing more spatial position key information.

6. An electronic device, comprising: The program instructions are executed by the processor to implement the steps corresponding to the planetary change detection method in any one of claims 1-3.

7. A computer storage medium having stored thereon computer program instructions, wherein the computer program instructions are executable by a computer to cause the computer to perform the method according to any one of claims 1 to 6. The program instructions are executed by the processor to implement the steps corresponding to the planetary change detection method in any one of claims 1-3.

Images