A remote sensing image scene-level change detection method

Abstract

A remote sensing image scene level change detection method, comprising: 1) based on principal component analysis, extracting the first three principal component components of two period images, using a pre-trained VGG-16 to extract scene depth features, using change vector analysis and fuzzy C mean clustering to generate scene level pseudo change map 1; 2) based on decision tree, generating pixel level classification map of two period images, using classification comparison method to generate pixel level change map, and proposing pixel level to scene level conversion strategy, converting pixel level change map to scene level pseudo change map 2; 3) fusing pseudo change map 1 and pseudo change map 2, generating reliable change and non-change training samples; 4) using automatically selected samples to train ternary change detection network; 5) inputting all scene pairs into the trained network to generate scene level change detection result. The present application can effectively obtain the semantic level change of two period images, and provides a new method for dynamic monitoring of urban functional area change.

Description

Technical Field

[0001] This invention relates to the field of remote sensing mapping technology, and in particular to a method for detecting scene-level changes in remote sensing images. Background Technology

[0002] As my country enters a critical period of high-quality urbanization, drastic human activities such as urban redevelopment, infrastructure construction, and new urban area development have led to significant changes in urban structure, pattern, and function. Accurate monitoring and interpretation of these urban changes are of great significance for comprehensively assessing urbanization trends, optimizing national spatial structure, and supporting sustainable urban decision-making.

[0003] Traditional manual ground-based monitoring of urban changes is insufficient to meet the demands for real-time information. Remote sensing Earth observation enables long-term, large-scale, and periodic observation of the Earth's surface, revolutionizing the technology for acquiring urban change information. Based on the granularity of the analysis unit, change detection methods can be categorized into pixel-level, object-level, and scene-level methods. Compared to pixel-level and object-level methods, scene-level change detection methods can obtain higher-level semantic or functional changes, such as changes from industrial areas to residential areas, becoming a new research direction in the field of change detection.

[0004] Scene-level change detection methods mainly fall into two categories: traditional and deep learning-based. Traditional methods primarily utilize bag-of-words models or topic models to extract mid-level features from images for scene-level change detection. While more efficient, these methods fail to extract deep features from high-resolution images, potentially leading to poor performance in images with complex terrain features. Deep learning methods, due to their powerful deep feature extraction capabilities, have become the mainstream approach for scene change detection. However, existing deep learning-based scene change detection methods require large amounts of training samples, have low automation levels, and are time-consuming and labor-intensive. Therefore, developing a fully automated change detection method is urgently needed to more efficiently monitor scene-level changes. Summary of the Invention

[0005] This invention addresses the shortcomings of existing technologies by providing a method for detecting scene-level changes in remote sensing images. This method overcomes the problems of insufficient deep feature mining and low automation in existing scene change detection methods, achieving efficient and accurate extraction of scene changes from two image phases.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] A method for detecting scene-level changes in remote sensing images includes the following steps:

[0008] S1: For the same area, acquire two periods of remote sensing images, extract the first three principal component components of the two periods of images based on principal component analysis as parameters, input them into a pre-trained VGG-16 neural network to extract scene depth features of the two periods of images, and use change vector analysis and fuzzy C-means clustering to generate the first scene-level pseudo-change map.

[0009] S2: For the two remote sensing images in step S1, generate pixel-level classification maps of the two images based on the decision tree, and generate pixel-level change maps using the post-classification comparison method. Then, convert the pixel-level change maps into second scene-level pseudo-change maps using the pixel-level to scene-level conversion method.

[0010] S3: Fuse the first scene-level pseudo-change map and the second scene-level pseudo-change map to generate training samples that have changed and have not changed.

[0011] S4: Construct a ternary change detection network and train it using the samples from step S3;

[0012] S5: The trained ternary change detection network is used for other scene recognition to generate scene-level change detection results.

[0013] To optimize the above technical solution, the specific measures also include:

[0014] Further, in step S1, the specific content of generating the first scene-level pseudo-change map using the change vector analysis method and fuzzy C-means clustering is as follows:

[0015] x ( )

[0016] In the formula, In the first scene-level pseudo-change graph, the first... m OK n The scenario of the series; This indicates the first image in two phases calculated using the vector change analysis method. m OK n The difference feature values ​​of the scene are listed, and the difference image is formed by comparing the two images; and These are the intensity values ​​of two cluster centers in the differential image calculated using fuzzy C-means clustering, and < ; , , These represent the changing class, the uncertain class, and the unchanging class, respectively.

[0017] Furthermore, in step S2, the specific content of generating the pixel-level classification map of the two-phase images based on the decision tree is as follows:

[0018] The Normalized Differential Water Index (NDWI) is calculated for each pixel in the two images, and a threshold X1 is determined by histogram filtering. When the NDWI index of a pixel is greater than the threshold X1, the corresponding pixel is determined to be a water body; otherwise, it is a non-water body. In this way, water bodies and non-water bodies are segmented in the two images respectively.

[0019] For non-water pixels, the Normalized Differential Vegetation Index (NDVI) is calculated, and the threshold X2 is determined by the OTSU method. When the NDVI index of a pixel is greater than the threshold X2, the corresponding pixel is determined to be vegetation; otherwise, it is impermeable surface. In this way, vegetation and impermeable surface are segmented in the two phases of images respectively.

[0020] In summary, pixel-level classification images for the two periods of images were obtained.

[0021] Furthermore, in step S2, the specific content of converting the pixel-level change map into a second scene-level pseudo-change map using the pixel-level to scene-level conversion method is as follows:

[0022]

[0023] In the formula, In the second scene-level pseudo-change graph, the first... m OK n The scenario of the series; Represents the pixel change graph. m OK n The number of pixels that change in the scene; Indicates a custom threshold; These represent the number of rows and columns of all pixels in the scene, respectively. , These represent the changing class and the unchanging class, respectively.

[0024] Further, step S3 specifically involves: fusing the first scene-level pseudo-change map and the second scene-level pseudo-change map; if a scene is in the change class in both pseudo-change maps, then the scene is selected as a change scene; if a scene is in the invariant class in both pseudo-change maps, then the scene is selected as an invariant scene; otherwise, it is considered an uncertain class and ignored; thereby generating training scene samples of change and invariance.

[0025] Furthermore, the specific content of step S4 is as follows:

[0026] The ternary change detection network includes a late-stage fusion subnetwork and an early-stage fusion subnetwork;

[0027] The late-stage fusion sub-network is used to input the training scene samples into the first basic feature extraction module, process them, and then output them to the first Ghost multi-scale feature module for further processing to obtain the first feature value; simultaneously, the training scene samples are input into the second basic feature extraction module, processed them, and then output to the second Ghost multi-scale feature module for further processing to obtain the second feature value; based on the first and second feature values, a feature difference value is obtained, which is then input into the first global average pooling layer to obtain a feature value that reflects the probability of scene change, which is the sample change probability value of the late-stage fusion sub-network;

[0028] The early fusion sub-network is used to input the training scene samples into the third basic feature extraction module, and after processing, output them to the third Ghost multi-scale feature module for further processing to obtain the third feature value. Then, it is input into the second global average pooling layer to obtain the feature value that can reflect the probability of scene change, which is the sample change probability value of the early fusion sub-network.

[0029] By performing feature concatenation on the sample change probability values ​​of the late-stage fusion subnetwork and the early-stage fusion subnetwork, the final sample change probability value of the entire ternary neural network is obtained; thus, the construction and training of the ternary change detection network are completed.

[0030] Furthermore, the first basic feature extraction module, the second basic feature extraction module, and the third basic feature extraction module each contain four convolutional layers and four pooling layers, wherein the number of convolutional kernels are 32, 32, 64, and 64, respectively, the kernel size is 3×3, and the activation function is ReLU;

[0031] The first Ghost multi-scale feature module, the second Ghost multi-scale feature module, and the third Ghost multi-scale feature module contain multiple Ghost convolutional layers with different convolutional kernels. Each Ghost convolutional layer outputs 64 features with a compression ratio of 2 and an activation function of ReLU.

[0032] Furthermore, the loss function of the ternary change detection network is: The specific calculation formula is as follows:

[0033]

[0034] In the formula, M Indicates the number of training scene samples; This represents the true value of the training sample; , and These represent the sample change probability values ​​output by the ternary change detection network, the early fusion sub-network, and the late fusion sub-network, respectively.

[0035] A computer-readable storage medium storing a computer program that causes a computer to perform the scene-level change detection method as described in any of the preceding claims.

[0036] An electronic device, characterized in that it comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the scene-level change detection method as described in any of the preceding claims.

[0037] The beneficial effects of this invention are: this application can effectively utilize the temporal features between two image periods and the depth features of each image period to achieve high-precision and high-efficiency scene-level change detection, better serving the dynamic monitoring of urban land use or functional areas. At the same time, it avoids the problems of existing deep learning-based scene change detection methods requiring a large number of training samples, having low automation, and being time-consuming and labor-intensive. Attached Figure Description

[0038] Figure 1 This is a true-value diagram of the image of the study area and scene-level changes in an example of the present invention.

[0039] Figure 2 This is a schematic diagram of the overall technical solution of the present invention.

[0040] Figure 3 This is a schematic diagram of the decision tree for pixel-level classification in this invention.

[0041] Figure 4 This is a schematic diagram of the fusion method of the two pre-detection pseudo-change maps of the present invention.

[0042] Figure 5 This is a schematic diagram of the lightweight multi-scale feature extraction module in an embodiment of the present invention.

[0043] Figure 6 This is a schematic diagram illustrating the change detection results of the method in this embodiment of the invention and the change detection results of other comparative methods. Detailed Implementation

[0044] The invention will now be described in further detail with reference to the accompanying drawings.

[0045] The example study area of ​​this invention is a part of Nanjing City, and the true values ​​of the two-phase imagery and scene-level changes are as follows: Figure 1 As shown. The two images are 20480 pixels × 20480 pixels in size, with a spatial resolution of 0.5 meters, and include four bands: blue, green, red, and near-infrared.

[0046] Figure 2 The flowchart shows a method for automatic scene-level change detection in remote sensing images based on knowledge-guided sample selection and a ternary neural network. The method of this invention mainly includes the following steps, which are described in detail below.

[0047] Step 1: Extract the first three principal component components (elements of the components are different bands) from the two images based on principal component analysis. Use pre-trained VGG-16 to extract scene depth features. Generate scene-level pseudo-changes using change vector analysis and fuzzy C-means clustering. Figure 1 The specific calculation method is as follows:

[0048] x ( )

[0049] in, Represents the change class, Represents the unchanging class. Represents the uncertain class, The first calculation using the vector variation method row and number The difference feature values ​​of the scenarios in the column, and ( < ) are the two class center strength values ​​calculated by fuzzy C-means clustering.

[0050] Step 2: Generate pixel-level classification maps for the two image periods based on the decision tree, generate pixel-level change maps using a post-classification comparison method, and propose a pixel-level to scene-level transformation strategy to convert the pixel-level change maps into scene-level pseudo-change maps. Figure 2 The conversion strategy is defined as follows:

[0051]

[0052] In the formula, for The number of pixels that change. Indicates a custom threshold; These represent the number of rows and columns of all pixels in the scene, respectively. , These represent the changing and unchanged classes, respectively. The instance study area is determined through parameter tuning. It is 0.25.

[0053] The decision tree is constructed based on the Normalized Difference Vegetation Index (NDVI) and the Normalized Difference Water Index (NDWI), specifically as follows: Figure 3 As shown.

[0054] Step 3, merge pseudo-changes Figure 1 and pseudo-changes Figure 2This generates reliable training samples of both changed and unchanged scenes. The fusion method is as follows: if a scene is in the changed class in both pseudo-change maps, then that scene is selected as a changed scene; if a scene is in the unchanged class in both pseudo-change maps, then that scene is selected as a unchanged scene; the remaining scenes are considered to be indeterminate. Specifically, as follows... Figure 4 As shown.

[0055] Step 4: Train the ternary change detection network using automatically selected samples. The network structure is described below:

[0056] 1) The three-element scene change detection network consists of a late fusion sub-network and an early fusion sub-network. Each sub-network includes a basic feature extraction module and a lightweight multi-scale feature extraction module. The basic feature extraction module consists of four convolutional layers and four pooling layers, with 32, 32, 64, and 64 kernels respectively, all of size 3×3, and ReLU activation function. The lightweight multi-scale feature extraction module combines Ghost convolutions with different kernels and pooling operations with different pooling methods. Each Ghost convolutional layer outputs 64 features with a compression ratio of 2 and ReLU activation function. The specific structure is as follows: Figure 5 As shown.

[0057] 2) The network employs a deep supervision strategy to calculate the loss function. The specific calculation method is as follows:

[0058]

[0059] in, M It is the number of training samples. It is the true value of the sample. , and These are the sample change probability values ​​output by the three-dimensional neural network, the early fusion sub-network, and the late sub-network, respectively.

[0060] Step 5: Input all scene pairs into the trained network to generate scene-level change detection results. To better illustrate the advantages of this invention, ten methods were compared. The detection results of different methods in the example study area are as follows: Figure 6 As shown in Table 1, the detection accuracy is as follows. It can be observed that the present invention can better detect scene-level changes, with both the F1 and OA scores of the detection results being higher than those of the other comparative methods.

[0061] Table 1

[0062]

[0063] The ternary change detection network of this application is a hybrid network that integrates an early fusion subnetwork and a late fusion subnetwork, making full use of the temporal features between the two images and the depth features of each image.

[0064] It should be noted that the terms such as "upper", "lower", "left", "right", "front", and "back" used in the invention are only for clarity of description and are not intended to limit the scope of the invention. Changes or adjustments to their relative relationships, without substantially altering the technical content, should also be considered within the scope of the invention.

[0065] The above are merely preferred embodiments of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should be considered within the scope of protection of the present invention.

Claims

1. A method for detecting scene-level changes in remote sensing images, characterized in that, Includes the following steps: S1: For the same area, acquire two periods of remote sensing images, extract the first three principal component components of the two periods of images based on principal component analysis as parameters, input them into a pre-trained VGG-16 neural network to extract scene depth features of the two periods of images, and use change vector analysis and fuzzy C-means clustering to generate the first scene-level pseudo-change map. S2: For the two remote sensing images in step S1, generate pixel-level classification maps of the two images based on the decision tree, and generate pixel-level change maps using the post-classification comparison method. Then, convert the pixel-level change maps into second scene-level pseudo-change maps using the pixel-level to scene-level conversion method. S3: Fuse the first scene-level pseudo-change map and the second scene-level pseudo-change map to generate training scene samples that have changed and have not changed. S4: Construct a ternary change detection network and train it using the samples from step S3; the specific content of step S4 is as follows: The ternary change detection network includes a late-stage fusion subnetwork and an early-stage fusion subnetwork; The late-stage fusion sub-network is used to input the training scene samples into the first basic feature extraction module, process them, and then output them to the first Ghost multi-scale feature module for further processing to obtain the first feature value; simultaneously, the training scene samples are input into the second basic feature extraction module, processed, and then output to the second Ghost multi-scale feature module for further processing to obtain the second feature value; based on the first and second feature values, a feature difference value is obtained, which is then input into the first global average pooling layer to obtain a feature value that reflects the probability of scene change, which is the sample change probability value of the late-stage fusion sub-network; The early fusion sub-network is used to input the training scene samples into the third basic feature extraction module, and after processing, output them to the third Ghost multi-scale feature module for further processing to obtain the third feature value. Then, it is input into the second global average pooling layer to obtain the feature value that can reflect the probability of scene change, which is the sample change probability value of the early fusion sub-network. The sample change probability values ​​of the late fusion subnetwork and the early fusion subnetwork are concatenated to obtain the final sample change probability value of the entire ternary neural network; in summary, the construction and training of the ternary change detection network are completed. S5: The trained ternary change detection network is used for other scene recognition to generate scene-level change detection results.

2. The method for detecting scene-level changes in remote sensing images according to claim 1, characterized in that, In step S1, the specific content of generating the first scene-level pseudo-change map using the change vector analysis method and fuzzy C-means clustering is as follows: x ( ) In the formula, In the first scene-level pseudo-change graph, the first... m OK n The scenario of the series; This indicates the first image in two phases calculated using the vector change analysis method. m OK n The difference feature values ​​of the scene are listed, and the difference image is formed by comparing the two images; and These are the intensity values ​​of two cluster centers in the differential image calculated using fuzzy C-means clustering, and < ; , , These represent the changing class, the uncertain class, and the unchanging class, respectively.

3. The method for detecting scene-level changes in remote sensing images according to claim 1, characterized in that, In step S2, the specific content of generating pixel-level classification maps for the two-phase images based on the decision tree is as follows: The Normalized Differential Water Index (NDWI) is calculated for each pixel in the two images, and a threshold X1 is determined by histogram filtering. When the NDWI index of a pixel is greater than the threshold X1, the corresponding pixel is determined to be a water body; otherwise, it is a non-water body. In this way, water bodies and non-water bodies are segmented in the two images respectively. For non-water pixels, the Normalized Differential Vegetation Index (NDVI) is calculated, and the threshold X2 is determined by the OTSU method. When the NDVI index of a pixel is greater than the threshold X2, the corresponding pixel is determined to be vegetation; otherwise, it is impermeable surface. In this way, vegetation and impermeable surface are segmented in the two phases of images respectively. In summary, pixel-level classification images for the two periods of images were obtained.

4. The method for detecting scene-level changes in remote sensing images according to claim 1, characterized in that, In step S2, the specific content of converting the pixel-level change map into a second scene-level pseudo-change map using the pixel-level to scene-level conversion method is as follows: In the formula, In the second scene-level pseudo-change graph, the first... m OK n The scenario of the series; Represents the pixel change graph. m OK n The number of pixels that change in the scene; Indicates a custom threshold; These represent the number of rows and columns of all pixels in the scene, respectively. , These represent the changing class and the unchanging class, respectively.

5. The method for detecting scene-level changes in remote sensing images according to claim 1, characterized in that, The specific content of step S3 is as follows: merge the first scene-level pseudo-change map and the second scene-level pseudo-change map; if a scene is in the change class in both pseudo-change maps, then the scene is selected as a change scene; if a scene is in the invariant class in both pseudo-change maps, then the scene is selected as an invariant scene; otherwise, it is regarded as an uncertain class and ignored; thereby generating training scene samples of change and invariance.

6. The method for detecting scene-level changes in remote sensing images according to claim 1, characterized in that, The first, second, and third basic feature extraction modules each contain four convolutional layers and four pooling layers, with the number of convolutional kernels being 32, 32, 64, and 64, respectively. The kernel size is 3×3, and the activation function is ReLU. The first Ghost multi-scale feature module, the second Ghost multi-scale feature module, and the third Ghost multi-scale feature module contain multiple Ghost convolutional layers with different convolutional kernels. Each Ghost convolutional layer outputs 64 features with a compression ratio of 2 and an activation function of ReLU.

7. The method for detecting scene-level changes in remote sensing images according to claim 1, characterized in that, The loss function of the ternary change detection network is: The specific calculation formula is as follows: In the formula, M Indicates the number of training scene samples; This represents the true values ​​of the training scenario samples; , and These represent the sample change probability values ​​output by the ternary change detection network, the early fusion sub-network, and the late fusion sub-network, respectively.

8. A computer-readable storage medium storing a computer program, characterized in that, The computer program causes the computer to execute the scene-level change detection method as described in any one of claims 1-7.

9. An electronic device, characterized in that, include: The memory, the processor, and the computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the scene-level change detection method as described in any one of claims 1-7.

Images