A deep learning-based multi-temporal remote sensing image automatic change detection method

Abstract

The application discloses a kind of based on deep learning multi-temporal remote sensing image automatic change detection method, comprising the following steps: collecting target area obtained remote sensing image data in different period, remote sensing image data is standardized, obtains standardized multi-temporal remote sensing image dataset;Based on standardized multi-temporal remote sensing image dataset, using semantic segmentation network carries out ground object class segmentation, obtains ground object semantic prior graph;Standardized multi-temporal remote sensing image dataset is input into improved U type twin network, and outputs change detection probability graph and binary change mask;Based on change detection probability graph and binary change mask, attribute discrimination is carried out to change region in combination with ground object semantic prior graph, and change region result is output;Change region result is integrated into geographic information service platform.The application uses improved U type twin network, realizes the automatic change detection of multi-temporal remote sensing image.

Description

Technical Field

[0001] This invention relates to the field of intelligent processing technology for remote sensing images, and in particular to an automatic change detection method for multi-temporal remote sensing images based on deep learning. Background Technology

[0002] Existing multi-temporal remote sensing image change detection techniques mainly include traditional spectral differencing, principal component analysis, and statistical model-based methods. These methods typically rely on pixel-level brightness or spectral changes for discrimination, making them susceptible to interference from sensor noise, ground cover changes, and atmospheric conditions, and thus difficult to accurately identify ground cover changes in complex scenes. With the development of deep learning technology, convolutional neural networks, U-Net structures, and Siamese network methods have been widely introduced into the field of remote sensing image change detection. Some studies have achieved automatic identification of changed areas by extracting features from multi-temporal images and performing feature differencing, thus improving the intelligence level of detection.

[0003] However, most existing deep learning methods encode images from different time phases using a uniform structure, lacking adaptive modeling of semantic differences in ground features across different time phases, and failing to address diverse scenarios involving both abrupt and gradual changes. Regarding fusion strategies, traditional methods typically employ only direct feature differencing or concatenation, unable to dynamically adjust to the complexity of changes in different ground feature categories, resulting in insufficient accuracy in change detection in areas with mixed distributions of artificial and natural ground features. Furthermore, existing methods generally neglect the guiding role of prior semantic knowledge of ground features in feature extraction and change discrimination, lacking innovative mechanisms to integrate prior knowledge of ground features with deep network features at the feature level, thus affecting the model's ability to discriminate changes in ground feature categories and boundary regions.

[0004] Therefore, how to provide an automatic change detection method for multi-temporal remote sensing images based on deep learning is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] One objective of this invention is to propose an automatic change detection method for multi-temporal remote sensing images based on deep learning. This invention applies an improved U-shaped twin network, semantic prior modulation of ground features, and a category-adaptive feature fusion mechanism, and details the automatic detection and classification process of changed regions in multi-temporal remote sensing images. The method can fully integrate semantic information of ground features, guiding the network to distinguish the different change characteristics of artificial and natural ground features, achieving intelligent identification of the spatial attributes and dominant ground feature categories of changed regions. It possesses advantages such as high detection accuracy, strong adaptability, and structured output of results.

[0006] An automatic change detection method for multi-temporal remote sensing images based on deep learning according to an embodiment of the present invention includes the following steps:

[0007] Collect remote sensing image data of the target area at different times, and perform standardization processing on the remote sensing image data to obtain a standardized multi-temporal remote sensing image dataset.

[0008] Based on a standardized multi-temporal remote sensing image dataset, a semantic segmentation network is used to segment ground cover categories and obtain a semantic prior map of ground covers.

[0009] The standardized multi-temporal remote sensing image dataset is input into the improved U-shaped twin network for feature extraction. The semantic prior map of ground features is fused with the feature maps of each layer to output a change detection probability map and a binary change mask.

[0010] Based on the change detection probability map and binary change mask, combined with the semantic prior map of ground features, the attribute of the change area is determined and the change area result is output.

[0011] By integrating the results of the changed areas into the geographic information service platform, the spatial display, comparative analysis, and online publication and application of the changed areas can be realized.

[0012] Optionally, the standardized multi-temporal remote sensing image dataset processing includes standardized remote sensing image data of each temporal phase that have undergone acquisition, radiometric correction, geometric correction, noise suppression, coordinate uniform projection, and spatial resolution resampling.

[0013] Optionally, the step of using a semantic segmentation network to segment land cover categories based on a standardized multi-temporal remote sensing image dataset to obtain a semantic prior map of land covers specifically includes:

[0014] Standardized remote sensing image data is selected from the standardized multi-temporal remote sensing image dataset and used as input data for ground feature semantic segmentation processing;

[0015] The standardized remote sensing image data is normalized to obtain the normalized standardized remote sensing image data.

[0016] Normalized and standardized remote sensing image data is input into a semantic segmentation network based on a convolutional neural network structure. Feature extraction and category discrimination are performed on each pixel of the input image, and a land cover category probability map is output.

[0017] Apply pixel-by-pixel maximum probability classification to the land cover category probability map, assign each pixel to the land cover category with the highest probability, and generate a land cover category label map.

[0018] The generated land cover category label map maintains a one-to-one correspondence with the standardized remote sensing image data in space, and is stored as a semantic prior map of land covers.

[0019] Optionally, the step of inputting the multi-temporal remote sensing image dataset into the improved U-shaped twin network and outputting a change detection probability map and a binary change mask includes:

[0020] Based on a multi-temporal remote sensing image dataset, standardized remote sensing images from different temporal phases are input into an improved U-shaped twin network. The improved U-shaped twin network includes two U-Net branches with shared parameters, a feature fusion layer, and a change discrimination decoding layer.

[0021] The two U-Net branches with shared parameters receive standardized remote sensing images of the corresponding time phase as input. Each U-Net branch includes multiple encoder layers, multiple decoder layers, and skip connection structures.

[0022] A ground feature semantic prior feature modulation module is embedded after each decoder layer of the first U-Net branch, and the feature map of the current decoder layer is modulated by conditional normalization;

[0023] A ground feature semantic prior feature modulation module is embedded after each encoder layer in the second U-Net branch, and the feature map of the current encoder layer is modulated through an attention mechanism;

[0024] The decoder multi-scale feature map output from the first U-Net branch and the encoder multi-scale feature map output from the second U-Net branch are input into the feature fusion layer to obtain change-sensitive features.

[0025] The change-sensitive features are input into the change discrimination decoding layer, the change-sensitive features are restored to the original spatial resolution, and the change detection probability map and binary change mask are output.

[0026] Optionally, embedding a ground feature semantic prior feature modulation module after each decoder layer of the first U-Net branch to modulate the decoder feature map specifically includes:

[0027] In each decoder layer of the first U-Net branch, the land cover semantic prior map input to the first U-Net branch is spatially aligned with the feature map of the current decoder layer;

[0028] The spatially aligned ground feature semantic prior map is used as the modulation condition input to the ground feature semantic prior feature modulation module. Conditional normalization is performed on the feature map of the current decoder layer to obtain the decoder feature map after ground feature semantic modulation.

[0029] The decoder feature map after semantic modulation of ground features is used as the output of the current decoder layer for use by the next level decoder layer or skip connection structure.

[0030] Repeat the above modulation process for all decoder layers until all decoder layers have been processed, and finally output the decoder multi-scale feature map after modulation of all ground feature semantic prior features.

[0031] Optionally, embedding a ground feature semantic prior feature modulation module after each encoder layer of the second U-Net branch to modulate the encoder feature map specifically includes:

[0032] In each encoder layer of the second U-Net branch, the land cover semantic prior map input from the second U-Net branch is spatially aligned with the feature map of the current encoder layer;

[0033] The spatially aligned semantic prior map of ground features is concatenated with the feature map of the current encoder layer in the channel dimension. The concatenation result is input into the convolution operation and activated by Sigmoid to generate a spatial attention weight map.

[0034] The feature map of the current encoder layer is weighted element-wise using a spatial attention weight map to obtain the encoder feature map after attention modulation.

[0035] The attention-modulated encoder feature map is used as the output of the current encoder layer for use by the next encoder layer or skip connection structure.

[0036] Repeat the above attention modulation process for all encoder layers until all encoder layers have been processed, and finally output the encoder multi-scale feature map after all land cover semantic prior attention modulation.

[0037] Optionally, the step of inputting the decoder multi-scale feature map output from the first U-Net branch and the encoder multi-scale feature map output from the second U-Net branch into the feature fusion layer to obtain change-sensitive features specifically includes:

[0038] The decoder multi-scale feature map output from the first U-Net branch and the encoder multi-scale feature map output from the second U-Net branch are input into the feature fusion layer;

[0039] A judgment and adjustment module is set in the feature fusion layer. The judgment and adjustment module automatically selects the fusion operation mode based on the land cover category information of the land cover semantic prior map at the current spatial location.

[0040] When the semantic prior map of local features is classified as artificial features at the current spatial location, the feature fusion layer generates a feature difference map by subtracting the feature maps from the two branches element by element at the current spatial location.

[0041] When the category of the local land cover semantic prior map is natural land cover at the current spatial location, the feature fusion layer uses a method of splicing the two branch output feature maps in the feature dimension to generate a spliced ​​feature map, or uses a method of calculating the distance between feature vectors to generate a distance feature map;

[0042] The results obtained by dynamically selecting the fusion method by the judgment and adjustment module constitute the change-sensitive features, which include feature difference map, feature stitching map and feature distance map.

[0043] Optionally, the step of inputting change-sensitive features into the change discrimination decoding layer and outputting a change detection probability map and a binary change mask includes:

[0044] The change-sensitive features obtained by the judgment and adjustment module are input into the change discrimination decoding layer;

[0045] The change-discrimination decoding layer consists of multiple decoding units, each of which is composed of an upsampling layer, a convolutional layer, an activation function, and a skip connection structure.

[0046] After undergoing multi-layer upsampling and skip connections, the change-sensitive features are mapped to a change detection probability map through a single convolution operation.

[0047] The change detection probability map is subjected to binary segmentation using a set threshold to obtain a binary change mask. Each pixel value of the binary change mask is 0 or 1, where 0 indicates no change and 1 indicates a change.

[0048] Optionally, the step of performing attribute discrimination on the changed area based on the change detection probability map and binary change mask, combined with the semantic prior map of ground features, and outputting the changed area result specifically includes:

[0049] Connected component extraction is performed on the binary transformation mask. All spatially connected transformation regions are identified by cell adjacency relationship, and a unique number is assigned to each transformation region. At the same time, the cell coordinates of the transformation region are recorded.

[0050] Extract the spatial attributes of each changing region, including spatial location, region boundary outline, and actual area;

[0051] Based on the semantic prior map of land features, the distribution of land feature categories of all pixels in each area of ​​change is statistically analyzed, and the land feature category with the most pixels is determined as the dominant category;

[0052] By combining the semantic prior maps of land features in different time phases, the land feature categories at the same spatial location in the two time phases are compared to determine the dominant category change type of the changed area;

[0053] Output the spatial location, boundary outline, actual area, dominant land cover category, and dominant category change type of each changed area to form a complete change area result.

[0054] Optionally, integrating the changed area results into the geographic information service platform specifically includes:

[0055] The results for each changed region are standardized and structured to generate structured data for the changed regions.

[0056] The structured data of the changed areas are uploaded to the geographic information service platform and spatially overlaid with the basic geographic information data to generate spatial display results;

[0057] Based on the spatial display results, the geographic information service platform provides attribute query, spatial measurement, layer switching and statistical analysis functions to generate comparative analysis results.

[0058] Based on the comparative analysis results, configure the network interface or application programming interface to publish the structured data of the changed area online;

[0059] The structured data of the changed areas are dynamically updated, version managed, and access controlled on the geographic information service platform.

[0060] The beneficial effects of this invention are:

[0061] This invention achieves refined identification of ground feature changes in multi-temporal remote sensing images by introducing semantic prior information about ground features and employing a branch-asynchronous feature modulation mechanism in a U-shaped twin network structure. By embedding semantic prior modulation modules into the encoder or decoder layers of different branches, the feature extraction method can be adaptively adjusted according to the ground feature category, thereby improving the sensitivity to changes in different types of ground features. This structure overcomes the shortcomings of traditional deep learning methods in discerning the diversity of change types and complex scenes.

[0062] In the feature fusion stage, this invention employs a dynamic selection strategy based on feature category discrimination, employing differential, stitching, or distance calculation to achieve targeted processing of change patterns for both artificial and natural features. The feature fusion method can be flexibly adjusted according to actual feature change attributes, improving the accuracy of change detection in areas with mixed multi-category features. Combined with connectivity analysis and attribute discrimination processes for changed areas, the invention provides structured output of the spatial location, outline, area, dominant feature category, and change type of the changed area, facilitating further application of the results in geographic information platforms.

[0063] The method of this invention integrates the change detection results with a geographic information service platform through an end-to-end automated change detection process using multi-temporal remote sensing images, supporting spatial display, comparative analysis, and online data publishing. The overall solution enhances the automation and intelligence of remote sensing image change detection, providing strong data support for high-precision monitoring and intelligent management of ground feature changes in complex scenarios. Attached Figure Description

[0064] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0065] Figure 1 This is a flowchart of an automatic change detection method for multi-temporal remote sensing images based on deep learning proposed in this invention;

[0066] Figure 2 This is a schematic diagram of the improved U-shaped twin network structure in the multi-temporal remote sensing image automatic change detection method based on deep learning proposed in this invention;

[0067] Figure 3 This is a schematic diagram of the feature fusion process based on semantic priors of ground features in the multi-temporal remote sensing image automatic change detection method based on deep learning proposed in this invention. Detailed Implementation

[0068] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0069] refer to Figure 1-3 An automatic change detection method for multi-temporal remote sensing images based on deep learning includes the following steps:

[0070] Collect remote sensing image data of the target area at different times, and perform standardization processing on the remote sensing image data to obtain a standardized multi-temporal remote sensing image dataset.

[0071] Based on a standardized multi-temporal remote sensing image dataset, a semantic segmentation network is used to segment ground cover categories and obtain a semantic prior map of ground covers.

[0072] The standardized multi-temporal remote sensing image dataset is input into the improved U-shaped twin network for feature extraction. The semantic prior map of ground features is fused with the feature maps of each layer to output a change detection probability map and a binary change mask.

[0073] Based on the change detection probability map and binary change mask, combined with the semantic prior map of ground features, the attribute of the change area is determined and the change area result is output.

[0074] By integrating the results of the changed areas into the geographic information service platform, the spatial display, comparative analysis, and online publication and application of the changed areas can be realized.

[0075] In this embodiment, the key feature is that the remote sensing image data of the target area acquired at different times is standardized to obtain a standardized multi-temporal remote sensing image dataset, specifically including:

[0076] Collect raw remote sensing image data of the target area at different times. The raw remote sensing image data includes remote sensing image files of different time phases and their corresponding time labels and spatial coordinate information.

[0077] Radiometric correction is performed on the raw remote sensing image data for each time phase. The pixel values ​​of the image are corrected using radiometric calibration coefficients to obtain radiometrically corrected remote sensing image data.

[0078] Geometric correction is performed on the radiometrically corrected remote sensing image data for each time phase. The image data is then registered to the standard geographic coordinate system using geographic reference points and transformation models to obtain the geometrically corrected remote sensing image data.

[0079] Noise suppression processing is performed on the geometrically corrected remote sensing image data, and spatial filtering or denoising algorithms are used to eliminate abnormal noise and speckles in the image.

[0080] All temporal remote sensing image data are uniformly projected onto the same geographic coordinate system, and coordinate transformation parameters are used to achieve consistency of spatial reference.

[0081] Remote sensing image data of each time phase are resampled according to the set target spatial resolution to obtain standardized remote sensing image data with uniform spatial resolution;

[0082] The standardized remote sensing image data of each time phase after the above processing are saved as a multi-time phase remote sensing image dataset.

[0083] In this embodiment, the step of using a semantic segmentation network to segment land cover categories based on a standardized multi-temporal remote sensing image dataset to obtain a semantic prior map of land covers specifically includes:

[0084] Standardized remote sensing image data is selected from the standardized multi-temporal remote sensing image dataset. The standardized remote sensing image data includes remote sensing image files with uniform spatial resolution and uniform coordinate system, which are used as input data for ground feature semantic segmentation processing.

[0085] The standardized remote sensing image data is normalized by linearly mapping the pixel values ​​of the image to the [0,1] interval according to a preset interval, so as to obtain the normalized standardized remote sensing image data.

[0086] Normalized remote sensing image data is input into a semantic segmentation network based on a convolutional neural network structure. The semantic segmentation network includes multiple convolutional layers, pooling layers, skip connections, and upsampling layers. Feature extraction and category discrimination are performed on each pixel of the input image, and a land cover category probability map with spatial resolution consistent with the input image is output.

[0087] Apply pixel-by-pixel maximum probability classification to the land feature category probability map, assign each pixel to the land feature category with the highest probability, and generate a land feature category label map. Each pixel in the land feature category label map is assigned a unique category code. The land feature categories include buildings, roads, woodlands, water bodies, farmland, and bare land.

[0088] The generated land cover category label map and the standardized remote sensing image data are spatially maintained in a one-to-one correspondence, and stored as a semantic prior map of land cover.

[0089] In the multi-temporal remote sensing image change detection process, the semantic prior map of ground features is synchronously transmitted and called with the standardized remote sensing image data of the corresponding time phase, ensuring that the ground feature category information participates in feature fusion processing in real time in the feature modulation module of the backbone network.

[0090] In this embodiment, the step of inputting the multi-temporal remote sensing image dataset into the improved U-shaped twin network and outputting a change detection probability map and a binary change mask includes:

[0091] Based on a multi-temporal remote sensing image dataset, standardized remote sensing images from different temporal phases are input into an improved U-shaped twin network. The improved U-shaped twin network includes two U-Net branches with shared parameters, a feature fusion layer, and a change discrimination decoding layer.

[0092] Two U-Net branches with shared parameters receive standardized remote sensing images of the corresponding time phase as input. Each U-Net branch includes multiple encoder layers, multiple decoder layers, and skip connection structures. The encoder layer includes convolutional layers, activation functions, and pooling layers. The decoder layer includes deconvolutional layers, activation functions, and upsampling layers. The skip connection structures are used to concatenate or superimpose the features of each layer of the encoder with the features of the corresponding layer of the decoder.

[0093] A ground feature semantic prior feature modulation module is embedded after each decoder layer of the first U-Net branch. The ground feature semantic prior feature modulation module takes the ground feature semantic prior map input from the first branch and the feature map of the current decoder layer as input, and modulates the feature map of the current decoder layer through conditional normalization.

[0094] A ground feature semantic prior feature modulation module is embedded after each encoder layer of the second U-Net branch. The ground feature semantic prior feature modulation module takes the ground feature semantic prior map input from the second branch and the feature map of the current encoder layer as input, and modulates the feature map of the current encoder layer through an attention mechanism.

[0095] The decoder multi-scale feature map output from the first U-Net branch and the encoder multi-scale feature map output from the second U-Net branch are input into the feature fusion layer. The feature fusion layer uses feature difference and vector distance calculation to obtain change-sensitive features.

[0096] The change-sensitive features are input into the change discrimination decoding layer, which includes a multi-layer upsampling and skip connection structure. The change-sensitive features are restored to the original spatial resolution, and the change detection probability map and binary change mask are output. Each pixel value of the change detection probability map represents the probability that the pixel has changed, and each pixel value of the binary change mask represents the discrimination result of whether or not a change has occurred.

[0097] In this embodiment, the step of embedding a ground feature semantic prior feature modulation module after each decoder layer of the first U-Net branch to modulate the decoder feature map specifically includes:

[0098] In each decoder layer of the first U-Net branch, the land cover semantic prior map input to the first U-Net branch is spatially aligned with the feature map of the current decoder layer to ensure that the spatial dimension of the land cover semantic prior map and the feature map of the current decoder layer are consistent.

[0099] The spatially aligned ground feature semantic prior map is used as the modulation condition input to the ground feature semantic prior feature modulation module. Conditional normalization is then performed on the feature map of the current decoder layer to obtain the decoder feature map after ground feature semantic modulation.

[0100] ;

[0101] in. This is the decoder feature map with modulated ground object semantics output from the current decoder layer of the first U-Net branch. The scaling factor is generated based on the semantic prior map of ground features input from the first U-Net branch. This is the result of normalizing the feature map of the current decoder layer in the first U-Net branch. This is the feature map of the current decoder layer in the first U-Net branch. The bias factor is generated based on the semantic prior map of ground features input from the first U-Net branch. This is a semantic prior map of ground features;

[0102] The scaling factor is specifically a scaling parameter tensor obtained by transforming the geographic semantic prior map of the first U-Net branch through a learnable mapping function, wherein the spatial dimension is related to the features of the current decoder layer. Figure 1 This reflects the modulation of characteristic amplitude by land cover category;

[0103] The bias factor is specifically a bias parameter tensor obtained by transforming the land cover semantic prior map of the first U-Net branch through a learnable mapping function, with spatial dimension and features of the current decoder layer. Figure 1 This reflects the adaptive adjustment of land cover categories to feature offset;

[0104] In the above formula, , , , as well as All of them are feature tensors with the same spatial resolution and number of channels, and all represent feature strengths or normalized feature values ​​in the network. Therefore, the physical dimensions on both sides of the formula are consistent, which ensures the rationality and effectiveness of feature modulation operations.

[0105] The initial theoretical basis of the above formula comes from the normalization method in the field of deep learning, and its basic form is:

[0106] ;

[0107] in, and Based on this theory, and using learnable scaling and bias factors, this invention innovatively uses the semantic prior map of ground features as conditional input, and generates them respectively through learnable mapping. and And the original feature map of the current decoder layer is normalized. Spatial adaptive modulation is performed to achieve semantic perception and dynamic adjustment of feature maps, thereby improving the ability to distinguish changes in ground features in remote sensing images.

[0108] The decoder feature map after the ground feature semantic modulation is used as the output of the current decoder layer and used by the next level decoder layer or skip connection structure to realize the dynamic modulation of the feature decoding process by the ground feature semantic prior information.

[0109] The above modulation process is repeated for all decoder layers until all decoder layers are processed. The final output is a multi-scale feature map of the decoder modulated with all the semantic prior features of the ground objects. This map is used as the output feature of the first U-Net branch and is used for the calculation of change-sensitive features in the feature fusion layer.

[0110] In this embodiment, the step of embedding a ground feature semantic prior feature modulation module after each encoder layer of the second U-Net branch to modulate the encoder feature map specifically includes:

[0111] In each encoder layer of the second U-Net branch, the land cover semantic prior map input from the second U-Net branch is spatially aligned with the feature map of the current encoder layer, so that the land cover semantic prior map and the feature map of the current encoder layer have the same spatial resolution.

[0112] The spatially aligned semantic prior map of ground features is concatenated with the feature map of the current encoder layer along the channel dimension. The concatenated result is then input into a convolution operation and processed. Activate and generate a spatial attention weight map;

[0113] The feature map of the current encoder layer is element-wise weighted using a spatial attention weight map to obtain the attention-modulated encoder feature map:

[0114] ;

[0115] in, This is the encoder feature map after attention modulation. This is the feature map of the current encoder layer. For the semantic prior map of land features, This indicates concatenation along the channel dimension. For convolution operations, for Activation function This is element-wise multiplication;

[0116] The physical dimensions of all terms on both sides of the formula are characteristic tensors or numerical products of tensors. , , and These are all feature tensors corresponding to the spatial resolution and number of channels of the input image. The output is a dimensionless weighting coefficient. Operation guaranteed and Since they are completely consistent in physical dimensions, the dimensions on both sides of the formula are consistent, which conforms to the common sense and engineering implementation requirements of deep learning feature processing.

[0117] The initial theoretical basis of this formula is the attention mechanism in the field of deep learning. Based on this theory, this application uses the semantic prior map of ground features as a modulation condition, concatenates it with the feature map of the current encoder layer, and then inputs them together into the convolution operation. After the convolution extracts joint features, the results are then processed... The spatial attention weights are activated and generated, and finally multiplied element-wise with the feature map of the current encoder layer to achieve spatial adaptive feature enhancement guided by prior knowledge of land cover categories.

[0118] The attention-modulated encoder feature map is used as the output of the current encoder layer for use by the next level encoder layer or skip connection structure, realizing the dynamic modulation of the feature encoding process by the semantic prior information of ground objects.

[0119] The above attention modulation process is repeated for all encoder layers until all encoder layers are processed. The final output is the encoder multi-scale feature map after all land cover semantic prior attention modulation, which is used as the output feature of the second U-Net branch for the calculation of change-sensitive features in the feature fusion layer.

[0120] In this embodiment, the step of inputting the decoder multi-scale feature map output from the first U-Net branch and the encoder multi-scale feature map output from the second U-Net branch into the feature fusion layer to obtain change-sensitive features specifically includes:

[0121] The decoder multi-scale feature map output from the first U-Net branch and the encoder multi-scale feature map output from the second U-Net branch are input into the feature fusion layer;

[0122] A judgment and adjustment module is set in the feature fusion layer. The judgment and adjustment module automatically selects the fusion operation mode based on the land cover category information of the land cover semantic prior map at the current spatial location.

[0123] When the semantic prior map of local features is classified as artificial features (including but not limited to buildings and roads) at the current spatial location, the feature fusion layer generates a feature difference map by subtracting the two branch output feature maps element by element at the current spatial location, which is used to enhance the sensitivity to changes in abrupt features.

[0124] When the category of the local land cover semantic prior map is natural land cover (including but not limited to woodland, farmland, and water bodies) at the current spatial location, the feature fusion layer uses a splicing method of two branch output feature maps in the feature dimension to generate a spliced ​​feature map, or uses a distance calculation method between feature vectors to generate a distance feature map, which is used to enhance the ability to express the changes of slowly changing or gradually changing land cover.

[0125] The results obtained by dynamically selecting the fusion method through the above-mentioned judgment and adjustment module constitute change-sensitive features. The change-sensitive features include feature difference maps generated by subtracting feature maps of ground objects after semantic modulation of different time phases element by element; feature stitching maps generated by stitching feature vectors of different time phases along the feature channel dimension; and feature distance maps generated by calculating the distance between feature vectors of different time phases.

[0126] In this embodiment, the step of inputting change-sensitive features into the change discrimination decoding layer and outputting a change detection probability map and a binary change mask includes:

[0127] The change-sensitive features obtained by the judgment and adjustment module, including feature difference map, feature stitching map, and feature distance map, are input to the change discrimination decoding layer.

[0128] The change discrimination decoding layer includes multiple decoding units. Each decoding unit consists of an upsampling layer, a convolutional layer, an activation function, and a skip connection structure. The upsampling layer is used to gradually improve the spatial resolution of the feature map. The convolutional layer is used to further extract change-related features. The skip connection structure is used to concatenate or add change-sensitive features with the high-resolution features of the corresponding layer of the encoder to enhance spatial details and change boundary information.

[0129] After undergoing multi-layer upsampling and skip connections, the change-sensitive features are mapped to a change detection probability map through a single convolution operation. Each pixel value in the change detection probability map is a real number between [0,1], representing the probability value of a change in the corresponding pixel.

[0130] The change detection probability map is subjected to binary segmentation using a set threshold to obtain a binary change mask. Each pixel value of the binary change mask is 0 or 1, where 0 indicates no change and 1 indicates a change.

[0131] The output change detection probability map and binary change mask serve as the final result of automatic change detection for multi-temporal remote sensing images.

[0132] In this embodiment, the step of performing attribute discrimination on the changed area based on the change detection probability map and the binary change mask, combined with the semantic prior map of ground features, and outputting the changed area result specifically includes:

[0133] Connected component extraction is performed on the binary change mask output by the change discrimination decoding layer. All spatially connected change regions are identified by cell adjacency relationship, and a unique number is assigned to each change region. At the same time, the cell coordinates of each change region are recorded.

[0134] Extract the spatial attributes of each changing region. The spatial attributes include spatial location (represented by geographic coordinates of an circumscribed rectangle or closed polygon), region boundary outline (obtained through a boundary tracking algorithm), and actual area (calculated by combining the number of pixels in the region with spatial resolution).

[0135] Based on the semantic prior map of land features, the distribution of land feature categories of all pixels in each area of ​​change is statistically analyzed, and the land feature category with the most pixels is determined as the dominant category;

[0136] By combining the semantic prior maps of land features in different time phases, the land feature categories at the same spatial location in the two time phases are compared to determine the dominant category change type of the changed area;

[0137] Output the spatial location, boundary outline, actual area, dominant land cover category, and dominant category change type of each changed area to form a complete change area result.

[0138] In this embodiment, integrating the changed area results into the geographic information service platform specifically includes:

[0139] The results for each changed region are standardized and structured to generate structured data for the changed regions.

[0140] The structured data of the changed area is uploaded to the geographic information service platform and spatially overlaid with the basic geographic information data to generate spatial display results, realizing the spatial visualization of the changed area on the map interface;

[0141] Based on the spatial display results, the geographic information service platform provides attribute query, spatial measurement, layer switching and statistical analysis functions, generates comparative analysis results, and supports users to conduct in-depth analysis of the attribute information, spatiotemporal distribution and change trends of the changed areas.

[0142] Based on the comparative analysis results, configure network interfaces or application programming interfaces (APIs) to publish structured data of changed areas online, enabling online sharing and remote access of changed area data to government departments, enterprise users, or the general public;

[0143] The structured data of changed areas are dynamically updated, version-managed, and access-controlled on the geographic information service platform to ensure the timeliness, security, and traceability of the structured data of changed areas, and to realize the continuous application and service of multi-temporal remote sensing image change detection results.

[0144] Example 1:

[0145] To verify the feasibility of this invention in practice, it was applied to an annual remote sensing image change monitoring project led by the provincial geographic information authority. In recent years, the rapid pace of regional urbanization has placed higher demands on automation and precision in remote sensing image change monitoring in areas such as farmland protection, urban and rural construction, and ecological environment management. Based on multi-year remote sensing satellite and aerial imagery of the entire province, a five-year continuous change detection task covering the entire region was conducted through a geographic information platform. Previously, such tasks typically relied on manual visual interpretation or traditional image difference methods, which were not only extremely labor-intensive and inefficient but also difficult to accurately distinguish complex changes in various land cover types. This was particularly true in scenarios involving urban boundary expansion, rural homestead adjustments, and water receding, where traditional methods were prone to misjudgments and missed detections in terms of change details and spatial positioning.

[0146] In practical applications, the project team first collected over five years of temporal remote sensing imagery, with single images achieving a resolution of up to 1 meter and covering an area exceeding 180,000 square kilometers. All data underwent standardization processing, including coordinate unification, image registration, and format transcoding. Utilizing a deep learning-based automatic change detection method for multi-temporal remote sensing images proposed in this invention, semantic segmentation of land cover was performed on the remote sensing images for each year to obtain detailed prior maps of land cover categories. This prior information, along with the standardized images, was input into an improved U-shaped twin network, automatically completing feature extraction, feature fusion, change-sensitive feature generation, and automatic identification of changed areas. The output of the change detection probability map and binary mask was further analyzed through connected component analysis to extract the spatial location, boundary contours, area, and dominant land cover types of the changed areas. Finally, the change area results were integrated into a geographic information service platform to achieve spatial display, attribute query, and dynamic statistical analysis.

[0147] Under the implementation of the method of this invention, data processing and change detection of approximately 90,000 remote sensing images were completed, and the processing cycle was shortened by more than 80% compared to manual interpretation. Comparative evaluations were conducted using urban expansion, road construction, farmland conversion to non-agricultural uses, forest degradation, and water body changes as targets. The method of this invention achieved an average accuracy of 97.6% and a recall rate of 95.8% in detecting changes in major land cover types. Compared to traditional methods based on threshold difference or manual visual interpretation, the average accuracy was improved by approximately 7 percentage points, and the recall rate by nearly 9 percentage points. Feedback from actual operational departments shows that the solution of this invention is more sensitive to the detection of newly added building land, reduction of urban green space, and changes in the area of ​​key water bodies, with a lower false alarm rate. Simultaneously, the system supports batch automated data processing, greatly reducing the workload of technical personnel and enabling rapid data updates and dynamic releases.

[0148] The table below compares the detection performance of major land cover types in five consecutive years of remote sensing image change monitoring, including annual change area, model detection accuracy, recall rate, traditional method accuracy, traditional method recall rate, and average processing time for both methods. Data are derived from actual monitoring task results and operational system feedback from the geographic information authorities.

[0149] Table 1. Comparison of Annual Remote Sensing Changes of Major Land Cover Types

[0150] ;

[0151] As can be seen from the table above, the present invention demonstrates significant advantages over traditional change detection methods in key performance indicators of multi-temporal remote sensing image change detection. Specifically, in the annual change monitoring of five major land cover types—buildings, roads, farmland, forest land, and water bodies—the detection accuracy of the present invention's method is above 96.7%, with the detection accuracy for buildings and water bodies reaching 98.4% and 98.1%, respectively, higher than the 91.5% and 92.2% of traditional methods. Regarding recall, the present invention achieves a recall rate exceeding 94.9% for all land cover types, with some categories such as water bodies, forest land, and buildings reaching 97.5%, 96.4%, and 97.2%, respectively, while the recall rate of traditional methods is generally below 90%. This result indicates that the automatic change detection method based on deep learning and the modulation mechanism of land cover semantic prior features can more accurately capture and distinguish complex change patterns of various land cover types, improving the ability to identify small-scale, blurred-boundary, and multi-category mixed change areas.

[0152] Furthermore, in terms of processing efficiency, the average automatic processing time of the method of this invention is between 1.2 and 1.7 hours, which is more than 85% shorter than the average time of traditional manual visual interpretation or simple threshold difference-based methods. Taking building categories as an example, the traditional method takes an average of 12.8 hours to complete an annual full-area change detection, while the present invention only takes 1.7 hours. This not only significantly reduces manpower input but also enables rapid processing of large-scale batch data. Especially in the annual change monitoring business of the provincial geographic information service system, the efficiency and automation advantages of this invention provide a solid foundation for competent authorities to obtain information on changes in land features in a timely manner and support various business decisions.

[0153] Overall, the multi-temporal remote sensing image automatic change detection method proposed in this invention has fully demonstrated its advantages of high precision, high recall, and high efficiency in practical engineering applications. It has greatly improved the automation, intelligence, and scalability of change monitoring tasks, solved the problems of traditional methods in accurately identifying changes in complex mixed land features and the long processing cycle, and provided strong technical support for the acquisition and application of high-quality change data for geographic information service platforms.

[0154] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A deep learning based multi-temporal remote sensing image automatic change detection method, characterized in that, The method comprises the following steps: Collecting remote sensing image data of a target area obtained at different time periods, and performing standardization processing on the remote sensing image data to obtain a standardized multi-temporal remote sensing image data set; Based on the standardized multi-temporal remote sensing image data set, a semantic segmentation network is used for ground object category segmentation to obtain a ground object semantic prior map, which specifically comprises: Selecting standardized remote sensing image data from the standardized multi-temporal remote sensing image data set as input data for ground object semantic segmentation processing; Performing normalization processing on the standardized remote sensing image data to obtain normalized standardized remote sensing image data; Inputting the normalized standardized remote sensing image data into a semantic segmentation network based on a convolutional neural network structure to perform feature extraction and category discrimination on each pixel of the input image, and outputting a ground object category probability map; Applying pixel-by-pixel maximum probability classification to the ground object category probability map to assign each pixel to the corresponding ground object category with the highest probability to generate a ground object category label map; The generated ground object category label map is stored in a spatial one-to-one correspondence with the standardized remote sensing image data as a ground object semantic prior map; Inputting the standardized multi-temporal remote sensing image data set into an improved U-shaped twin network to perform feature extraction, fusing the ground object semantic prior map with each layer of feature maps, and outputting a change detection probability map and a binary change mask, which specifically comprises: Based on the multi-temporal remote sensing image data set, inputting standardized remote sensing images of different time periods into an improved U-shaped twin network, wherein the improved U-shaped twin network comprises two parameter-shared U-Net branches, a feature fusion layer, and a change discrimination decoding layer; The two parameter-shared U-Net branches respectively receive the standardized remote sensing images of the corresponding time periods as input, and each U-Net branch comprises multiple encoder layers, multiple decoder layers, and a skip connection structure; A ground object semantic prior feature modulation module is embedded after each decoder layer of the first U-Net branch to modulate the feature maps of the current decoder layer through conditional normalization; A ground object semantic prior feature modulation module is embedded after each encoder layer of the second U-Net branch to modulate the feature maps of the current encoder layer through an attention mechanism; The decoder multi-scale feature maps output by the first U-Net branch and the encoder multi-scale feature maps output by the second U-Net branch are input into the feature fusion layer to obtain change-sensitive features; The change-sensitive features fused by the judgment adjustment module are input into the change discrimination decoding layer to restore the change-sensitive features to the original spatial resolution, and output a change detection probability map and a binary change mask; The change discrimination decoding layer comprises multiple decoding units, each decoding unit being composed of an up-sampling layer, a convolutional layer, an activation function, and a skip connection structure; After the change-sensitive features are processed by multiple layers of up-sampling and skip connection, they are mapped into a change detection probability map through a convolutional operation; Based on the change detection probability map and the binary change mask, the attribute of the change region is discriminated in combination with the ground object semantic prior map to output a change region result; The change region result is integrated into a geographic information service platform to realize spatial display, comparative analysis, and online publishing and application of the change data.

2. The deep learning-based multi-temporal remote sensing image automatic change detection method according to claim 1, characterized in that, The standardized multi-temporal remote sensing image dataset processing includes various temporal standardized remote sensing image data processed through collection, radiation correction, geometric correction, noise suppression, coordinate unification projection and spatial resolution resampling. 3.The deep learning based multi-temporal remote sensing image automatic change detection method according to claim 1, characterized in that, The embedding of the landmark semantic prior feature modulation module after each decoder layer of the first U-Net branch specifically includes: At each decoder layer of the first U-Net branch, the landmark semantic prior map input into the first U-Net branch is spatially aligned with the feature map of the current decoder layer; The spatially aligned landmark semantic prior map is input into the landmark semantic prior feature modulation module as a modulation condition to perform conditional normalization on the feature map of the current decoder layer, so as to obtain the landmark semantic modulated decoder feature map; The landmark semantic modulated decoder feature map is taken as the output of the current decoder layer for use by the next decoder layer or the skip connection structure; The modulation process is repeated for all decoder layers, including spatial alignment, inputting the spatially aligned landmark semantic prior map into the landmark semantic prior feature modulation module as a modulation condition, conditional normalization and taking the modulated decoder feature map as the output of the decoder layer, until the processing of all levels of the decoder is completed, and finally the decoder multiscale feature map modulated by all landmark semantic prior features is output. 4.The deep learning based multi-temporal remote sensing image automatic change detection method of claim 1, wherein, The embedding of the landmark semantic prior feature modulation module after each encoder layer of the second U-Net branch specifically includes: At each encoder layer of the second U-Net branch, the landmark semantic prior map input into the second U-Net branch is spatially aligned with the feature map of the current encoder layer; The spatially aligned landmark semantic prior map and the feature map of the current encoder layer are spliced in the channel dimension, and the splicing result is input into convolution operation and activated by Sigmoid to generate a spatial attention weight map; The spatial attention weight map is used to perform element-by-element weighting on the feature map of the current encoder layer to obtain an attention modulated encoder feature map; The attention modulated encoder feature map is taken as the output of the current encoder layer for use by the next encoder layer or the skip connection structure; The attention modulation process is repeated for all encoder layers, including spatial alignment, channel dimension splicing, convolution operation, Sigmoid activation, element-by-element weighting and taking the attention modulated encoder feature map as the output of the current encoder layer, until the processing of all levels of the encoder is completed, and finally the encoder multiscale feature map modulated by all landmark semantic prior attentions is output.

5. The deep learning-based multi-temporal remote sensing image automatic change detection method according to claim 1, characterized in that, The input of the decoder multiscale feature map output by the first U-Net branch and the encoder multiscale feature map output by the second U-Net branch into the feature fusion layer to obtain change-sensitive features specifically includes: The input of the decoder multiscale feature map output by the first U-Net branch and the encoder multiscale feature map output by the second U-Net branch into the feature fusion layer; The judgment adjustment module is arranged in the feature fusion layer, and the judgment adjustment module automatically selects a fusion operation mode according to feature class information of the feature semantic priori map at a current spatial position; When the feature class information of the feature semantic priori map at the current spatial position is artificial feature, the feature fusion layer adopts a mode of element-by-element subtraction of the feature maps output by the two branches at the current spatial position to generate a feature difference map; When the feature class information of the feature semantic priori map at the current spatial position is natural feature, the feature fusion layer adopts a mode of splicing of the feature maps output by the two branches in the feature dimension to generate a spliced feature map, or adopts a distance calculation mode between the feature vectors to generate a distance feature map; The result obtained by dynamically selecting the fusion mode by the judgment adjustment module constitutes a change-sensitive feature, and the change-sensitive feature includes the feature difference map, the feature spliced map and the feature distance map. 6.The deep learning based multi-temporal remote sensing image automatic change detection method according to claim 1, characterized in that, The binary change mask is obtained by performing binary segmentation on the change detection probability map by using a set threshold, each pixel value of the binary change mask is 0 or 1, 0 represents no change, and 1 represents change.

7. The deep learning-based multi-temporal remote sensing image automatic change detection method according to claim 1, characterized in that, Based on the change detection probability map and the binary change mask, the attribute of the change region is identified in combination with the feature semantic priori map, and the change region result is output, which specifically includes: The binary change mask is subjected to connected domain extraction, all spatially connected change regions are identified by using pixel adjacency relationship, and each change region is assigned a unique number, and pixel coordinates of the change region are recorded; Spatial attributes of each change region are extracted, and the spatial attributes include spatial position, region boundary contour and actual area; According to the feature semantic priori map, the distribution of feature classes of all pixels in each change region is counted, and the feature class with the largest number of pixels is determined as the dominant class; In combination with the feature semantic priori maps of different time phases, the feature classes of the same spatial position in two time phases are compared to determine the change type of the dominant class of the change region; The spatial position, boundary contour, actual area, dominant feature class and change type of the dominant class of each change region are output to form a complete change region result. 8.The deep learning based multi-temporal remote sensing image automatic change detection method of claim 1, wherein, The change region result is integrated into a geographic information service platform, which specifically includes: Each change region result is subjected to standardized and structured processing to generate change region structured data; The change region structured data is uploaded to the geographic information service platform and is spatially superimposed with basic geographic information data to generate a spatial display result; Based on the spatial display result, attribute query, spatial measurement, layer switching and statistical analysis functions are provided in the geographic information service platform to generate a comparative analysis result; Based on the comparative analysis result, a network interface or an application programming interface is configured to online publish the change region structured data; The change region structured data is dynamically updated, version managed and permission controlled in the geographic information service platform.

Images