Remote sensing image fine object level change detection method and device

Abstract

The application provides a remote sensing image fine object-level change detection method and device, and belongs to the technical field of remote sensing image processing. The method comprises the following steps: selecting a change detection data set with front and rear time phase images and label data sets, and performing preprocessing; extracting multi-scale layered features from the preprocessed change detection data set, fusing the multi-scale layered features to obtain a fused feature map; finely refining the fused feature map to obtain high-dimensional feature representation; performing object-level change detection and instance-level segmentation processing on the high-dimensional features by using a unified network to simultaneously obtain object-level bounding box representation and fine boundary representation of the changed object; and performing network training of a hybrid loss function by using the obtained object-level bounding box representation and fine boundary representation of the changed object to output a change region detection result. The application avoids the fragmentation and inaccuracy of change detection edge information, and realizes fine change boundary.

Description

Technical Field

[0001] This invention relates to a method and apparatus for detecting refined object-level changes in remote sensing images, belonging to the field of remote sensing image processing technology. Background Technology

[0002] As a crucial task in Earth observation, change detection in remote sensing images has always been a research hotspot in the field, with its results widely applied in ecological environment monitoring, agricultural production, and military defense. However, compared to other remote sensing interpretation techniques (such as classification and recognition), change detection in remote sensing images involves a more complex process, higher requirements for missed and false detection rates, and greater challenges in terms of technical methods. In recent years, advancements in remote sensing imaging technology and progress in computer science have promoted the development of deep learning-based remote sensing change detection methods. These methods significantly reduce missed and false detection rates and error accumulation, and have become the mainstream change detection methods at present. In terms of detection granularity, current deep learning change detection methods can be divided into pixel-level and object-level methods.

[0003] Pixel-level change detection in remote sensing images uses individual pixels as detection units. It analyzes pixel spectral differences through pixel-by-pixel arithmetic operations to extract change features. Pixel-level change detection is currently the most mainstream deep learning-based change detection method. Recent methods such as FC-EF, FC-Siam-Conc, and FC-Siam-Diff apply fully convolutional, UNet, and Siamese neural structures to change detection, significantly improving detection speed. Methods like DASNet, SNUNet, USSFC, and SGSLN have achieved good detection accuracy on multiple datasets. However, in cases of changes across multiple land types, pixel-level change detection methods can only output the overall change area, making it difficult to distinguish the boundaries and specific extent of each changed object. For example, when some change areas appear in clusters, pixel-level change detection cannot distinguish object-level regions, only showing the location of pixel changes. This makes it difficult to identify specific objects within clusters of change areas, limiting the effectiveness of subsequent analyses such as land spatial planning.

[0004] Object-level change detection uses image patches or superpixels as input units to detect specific objects. Compared to pixel-level change detection, object-level change detection has higher discrimination accuracy for changed objects and lower requirements for cross-sectional image registration. However, existing object-level change detection methods can only output bounding boxes of changed regions, lacking a detailed description of the boundaries of changed objects, and their detection accuracy still lags behind pixel-level methods. Therefore, how to conduct refined object-level change detection and improve accuracy is currently the key to applying object-level change detection methods.

[0005] In recent years, the Transformer architecture has developed rapidly in natural language processing and computer vision, achieving significant success in multiple fields. In object detection, DINO, a Transformer (DEtectionTransformer, DETR) model similar to object detection, achieved the best result on the COCO object detection leaderboard for the first time. In image segmentation, MaskFormer and Mask2Former, employing a query-based Transformer architecture, unified image segmentation and mask classification, significantly improving the performance of image segmentation tasks. Against this backdrop, researchers have attempted to introduce the Transformer architecture into change detection, resulting in models such as TransUnetCD (combining the advantages of Transformer and Unet with strong generalization capabilities), BIT-CD (significantly reducing computational cost and parameter count), and ChangeFormer (better representing multi-scale long-distance details). However, object-level change detection models based on the Transformer architecture are still relatively few. Some deep learning-based object-level change detection methods still use pure convolutional object detection network structures (such as Faster R-CNN), and the object-level change detection results only output bounding boxes, leading to difficulties in distinguishing changed objects and a lack of refined description of change boundaries, resulting in insufficient accuracy of the changed region. Summary of the Invention

[0006] To address the aforementioned problems, this invention proposes a method and apparatus for detecting refined object-level changes in remote sensing images, which can solve the object-level localization problem in change detection.

[0007] The technical solution adopted by this invention to solve its technical problem is as follows:

[0008] In a first aspect, embodiments of the present invention provide a method for detecting refined object-level changes in remote sensing images, comprising the following steps:

[0009] Step 1: Select a change detection dataset containing before-and-after temporal images and label datasets, and preprocess the change detection dataset;

[0010] Step 2: Extract multi-scale hierarchical features from the preprocessed change detection dataset, and fuse the multi-scale hierarchical features to obtain a fused feature map;

[0011] Step 3: Refine the fused feature map to obtain a high-dimensional feature representation;

[0012] Step 4: Perform object-level change detection and instance-level segmentation processing on the high-dimensional features using a unified network, and obtain object-level bounding box representations and fine-grained boundary representations of the changed objects.

[0013] Step 5: Use the obtained object-level bounding box representation and the fine boundary representation of the changing object to train the network with a hybrid loss function, and output the change region detection results.

[0014] As one possible implementation of this embodiment, in step 1, the label dataset is a dataset of positive and negative samples of newly added construction land; the preprocessing of the change detection dataset is to enhance the change detection dataset by using a data augmentation method of randomly adding fog effects, randomly adding noise, randomly blurring, randomly grayscale, and contrast-limited adaptive histogram equalization.

[0015] As one possible implementation of this embodiment, step 2 includes:

[0016] Step 21: Extract multi-scale hierarchical features from the dual-temporal images using a residual network, and fuse them using a connection operation:

[0017]

[0018] in, and These represent the i-th layer features extracted from the preceding and following temporal images by the hierarchical temporal fusion module, where i = 1, 2, 3, 4, 5; Concatenate() represents the concatenation operation.

[0019] Step 22, connect the hierarchical features F bi Planarization into one-dimensional feature F hi :

[0020] F hi =Flatten(F bi (2)

[0021] Flatten() represents the flattening operation;

[0022] Step 23, for one-dimensional feature F hi Perform layered time fusion:

[0023] F e =Concatenate(F hi (3)

[0024] Among them, F e These are the features after hierarchical temporal fusion.

[0025] As one possible implementation of this embodiment, the extraction of multi-scale hierarchical features from dual-temporal images includes:

[0026] Perform a 7×7 convolution on the preceding and following time-phase images Image1 and Image2 respectively:

[0027]

[0028] The preceding and following temporal features F obtained from the convolution operation conv1 F conv2 The process proceeds sequentially through a batch normalization layer (BatchNorm), a ReLU activation layer, and a max pooling layer:

[0029]

[0030] The processed feature F′ conv1 and F′ conv2 The input is processed by the residual network to obtain the i-th layer features of the preceding and following temporal images.

[0031] As one possible implementation of this embodiment, step 3 includes:

[0032] Step 31: Add positional encoding to the feature map;

[0033] Step 32: The multi-scale feature maps are flattened into a sequence and processed through a series of Transformer coding layers;

[0034] Step 33: Select the top K features from the last layer of the encoder, perform position query and content query to initialize the decoder, and obtain the features to be processed;

[0035] Step 33: Use cross-attention mechanism and mask self-attention mechanism to process the features to be processed and obtain high-dimensional feature representation.

[0036] As one possible implementation of this embodiment, step 4 includes:

[0037] For high-dimensional features Q" m The bounding box representation of the changed object is output using the object-level change detection head:

[0038] Boxes = BoxRegression(Q″) m (14)

[0039] Classes=ClassClassification(Q″ m (15)

[0040] Here, BoxRegression() represents the bounding box regression operation, ClassClassification() represents the classification of the changed object, and Boxes and Classes represent the bounding box coordinates and class of the changed object, respectively;

[0041] For high-dimensional features Q" m Use split branches to generate fine-grained boundary representations for the modified objects:

[0042] Masks = MaskPrediction(Q″ m (16) Wherein, Q" m Represents high-dimensional features, Masks is the generated boundary representation of changes, and MaskPrediction() represents the mask prediction operation.

[0043] As one possible implementation of this embodiment, step 5 includes:

[0044] Step 51, calculate the localization loss for object-level change detection, wherein the localization loss L box Including minimum absolute value deviation loss (L1) and generalized crossover ratio loss (L... GIOU );

[0045] Step 52, calculate the classification loss (L) for object-level change detection using Focal loss. cls );

[0046] Step 53, calculate the masking loss used for the segmentation task, the masking loss including cross-entropy loss (L ce ) and Dice Loss (L dice );

[0047] Step 54: Linearly add the loss functions of localization loss, classification loss and masking loss to obtain the total loss function;

[0048] Step 55: Use the total loss function as the training loss function to train the object-level change detection model, and output the final trained model and the object-level change region detection results.

[0049] Secondly, an embodiment of the present invention provides a device for detecting refined object-level changes in remote sensing images, comprising:

[0050] The dataset acquisition module is used to select a change detection dataset containing before-and-after temporal images and label datasets, and to preprocess the change detection dataset.

[0051] The feature fusion module is used to extract multi-scale hierarchical features from the preprocessed change detection dataset and fuse the multi-scale hierarchical features to obtain a fused feature map.

[0052] The feature extraction module is used to refine the fused feature map and obtain a high-dimensional feature representation.

[0053] The boundary representation module is used to perform object-level change detection and instance-level segmentation processing on high-dimensional features using a unified network, while obtaining object-level bounding box representations and fine-grained boundary representations of changing objects.

[0054] The network training module is used to train the network using a hybrid loss function based on the obtained object-level bounding box representation and the fine boundary representation of the changing object, and outputs the change region detection results.

[0055] Thirdly, an embodiment of the present invention provides a computer device including a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the computer device is running, the processor communicates with the memory via the bus, and the processor executes the machine-readable instructions to perform the steps of the above-described method for detecting refined object-level changes in any remote sensing image.

[0056] Fourthly, embodiments of the present invention provide a storage medium storing a computer program, which, when run by a processor, executes the steps of the method for detecting refined object-level changes in any remote sensing image as described above.

[0057] The beneficial effects of the technical solutions of the embodiments of the present invention are as follows:

[0058] To address the shortcomings of existing change detection models, such as difficulty in distinguishing changed objects and lack of refined description of change boundaries, this invention proposes a refined object-level change detection method for remote sensing images. By constructing a unified framework for object-level change detection and segmentation based on Transformer, it simultaneously outputs refined boundary representations and bounding box representations of changed objects, more effectively comparing object-level and pixel-level detection accuracy. This method can accurately locate changed regions and easily distinguish each changed object within the changed region, successfully solving the object-level localization problem in change detection and avoiding the fragmentation and inaccuracy of change detection edge information, thus achieving refined change boundaries. This invention employs a Transformer structure, multi-head attention mechanism, and positional encoding structure to achieve dynamic attention weight allocation of features across different remote sensing image locations, effectively improving the contextual features between different regions of the remote sensing image. Building upon existing feature extraction networks, this invention effectively proposes a hierarchical temporal fusion module, forming a unified object-level change detection and segmentation framework based on advanced Transformer-based object detection networks, effectively improving the network's change detection accuracy. Attached Figure Description

[0059] Figure 1This is a flowchart illustrating a method for detecting refined object-level changes in remote sensing images according to an exemplary embodiment;

[0060] Figure 2 This is a schematic diagram of the structure of a remote sensing image fine-grained object-level change detection device according to an exemplary embodiment;

[0061] Figure 3 This is a schematic diagram illustrating a specific implementation of the present invention for detecting changing regions within a public dataset;

[0062] Figure 4 This is a schematic diagram of the basic structure of the residual unit;

[0063] Figure 5 This is a schematic diagram of the bottleneck structure of the residual unit;

[0064] Figure 6 This is a partial visualization of the change detection results of the method of the present invention. Detailed Implementation

[0065] To more clearly illustrate the technical features of the present invention, the present invention will be described in detail below through specific embodiments and in conjunction with the accompanying drawings.

[0066] like Figure 1 As shown in the figure, an embodiment of the present invention provides a method for detecting refined object-level changes in remote sensing images, comprising the following steps:

[0067] Step 1: Select a change detection dataset containing before-and-after temporal images and label datasets, and preprocess the change detection dataset;

[0068] Step 2: Extract multi-scale hierarchical features from the preprocessed change detection dataset, and fuse the multi-scale hierarchical features to obtain a fused feature map;

[0069] Step 3: Refine the fused feature map to obtain a high-dimensional feature representation;

[0070] Step 4: Perform object-level change detection and instance-level segmentation processing on the high-dimensional features using a unified network, and obtain object-level bounding box representations and fine-grained boundary representations of the changed objects.

[0071] Step 5: Use the obtained object-level bounding box representation and the fine boundary representation of the changing object to train the network with a hybrid loss function, and output the change region detection results.

[0072] As one possible implementation of this embodiment, in step 1, the label dataset is a dataset of positive and negative samples of newly added construction land; the preprocessing of the change detection dataset is to enhance the change detection dataset by using a data augmentation method of randomly adding fog effects, randomly adding noise, randomly blurring, randomly grayscale, and contrast-limited adaptive histogram equalization.

[0073] As one possible implementation of this embodiment, step 2 includes:

[0074] Step 21: Extract multi-scale hierarchical features from the dual-temporal images using a residual network, and fuse them using a connection operation:

[0075]

[0076] in, and These represent the i-th layer features extracted from the preceding and following temporal images by the hierarchical temporal fusion module, where i = 1, 2, 3, 4, 5; Concatenate() represents the concatenation operation.

[0077] Step 22, connect the hierarchical features F bi Planarization into one-dimensional feature F hi :

[0078] F hi =Flatten(F bi (2)

[0079] Flatten() represents the flattening operation;

[0080] Step 23, for one-dimensional feature F hi Perform layered time fusion:

[0081] F e =Concatenate(F hi (3)

[0082] Among them, F e These are the features after hierarchical temporal fusion.

[0083] As one possible implementation of this embodiment, the extraction of multi-scale hierarchical features from dual-temporal images includes:

[0084] Perform a 7×7 convolution on the preceding and following time-phase images Image1 and Image2 respectively:

[0085]

[0086] The features obtained from the convolution operation are then processed through a batch normalization layer, a ReLU activation function layer, and a max pooling layer.

[0087]

[0088] Feature F′ conv1 and F′ conv2 The input is processed by the residual network to obtain the i-th layer features of the preceding and following temporal images.

[0089] As one possible implementation of this embodiment, step 3 includes:

[0090] Step 31: Add positional encoding to the feature map;

[0091] Step 32: The multi-scale feature maps are flattened into a sequence and processed through a series of Transformer coding layers;

[0092] Step 33: Select the top K features from the last layer of the encoder, perform position query and content query to initialize the decoder, and obtain the features to be processed;

[0093] Step 33: Use cross-attention mechanism and mask self-attention mechanism to process the features to be processed and obtain high-dimensional feature representation.

[0094] As one possible implementation of this embodiment, step 4 includes:

[0095] For high-dimensional features Q" m The bounding box representation of the changed object is output using the object-level change detection head:

[0096] Boxes = BoxRegression(Q″) m (14)

[0097] Classes=ClassClassification(Q″ m (15)

[0098] Here, BoxRegression() represents the bounding box regression operation, ClassClassification() represents the classification of the changed object, and Boxes and Classes represent the bounding box coordinates and class of the changed object, respectively;

[0099] For high-dimensional features Q" m Use split branches to generate fine-grained boundary representations for the modified objects:

[0100] Masks = MaskPrediction(Q″ m (16) Wherein, Q"m Represents high-dimensional features, Masks is the generated boundary representation of changes, and MaskPrediction() represents the mask prediction operation.

[0101] As one possible implementation of this embodiment, step 5 includes:

[0102] Step 51, calculate the localization loss for object-level change detection, wherein the localization loss L box Including minimum absolute value deviation loss (L1) and generalized crossover ratio loss (L... GIOU );

[0103] Step 52, calculate the classification loss (L) for object-level change detection using Focal loss. cls );

[0104] Step 53, calculate the masking loss used for the segmentation task, the masking loss including cross-entropy loss (L ce ) and Dice Loss (L dice );

[0105] Step 54: Linearly add the loss functions of localization loss, classification loss and masking loss to obtain the total loss function;

[0106] Step 55: Use the total loss function as the training loss function to train the object-level change detection model, and output the final trained model and the object-level change region detection results.

[0107] like Figure 2 As shown in the figure, an embodiment of the present invention provides a device for detecting refined object-level changes in remote sensing images, comprising:

[0108] The dataset acquisition module is used to select a change detection dataset containing before-and-after temporal images and label datasets, and to preprocess the change detection dataset.

[0109] The feature fusion module is used to extract multi-scale hierarchical features from the preprocessed change detection dataset and fuse the multi-scale hierarchical features to obtain a fused feature map.

[0110] The feature extraction module is used to refine the fused feature map and obtain a high-dimensional feature representation.

[0111] The boundary representation module is used to perform object-level change detection and instance-level segmentation processing on high-dimensional features using a unified network, while obtaining object-level bounding box representations and fine-grained boundary representations of changing objects.

[0112] The network training module is used to train the network using a hybrid loss function based on the obtained object-level bounding box representation and the fine boundary representation of the changing object, and outputs the change region detection results.

[0113] like Figure 1 and Figure 3 As shown, the specific steps for detecting changing regions within a public dataset using this invention are as follows:

[0114] Step 1: Select a change detection dataset with before-and-after temporal images and labels, and preprocess the change detection dataset.

[0115] A large number of positive and negative samples of newly added construction land were collected to construct a sample set, which was then divided into a training set and a validation set. Various data augmentation methods were used to enhance the collected data, specifically including random fog effects, random noise, random blurring, random grayscale conversion, and contrast-limited adaptive histogram equalization.

[0116] Step 2: Extract multi-scale hierarchical features from the preprocessed change detection dataset, and fuse the multi-scale hierarchical features to obtain a fused feature map.

[0117] This invention designs a hierarchical temporal fusion module that extracts and fuses multi-scale features, then feeds the fused features into an encoder for multi-scale feature extraction and hierarchical temporal fusion. The specific implementation process of step 2 is as follows:

[0118] Step 2.1: Input the dual-temporal images into the backbone feature extraction network of the hierarchical temporal fusion module to extract multi-scale hierarchical features. The backbone feature extraction network adopts a residual network, and then the features are fused using a connection operation. The formula is as follows:

[0119]

[0120] in, and These represent the i-th layer features of the preceding and following temporal images extracted from the hierarchical temporal fusion module, respectively.

[0121] Step 2.2: Obtain the hierarchical features F bi Connect and planarize into one-dimensional features to form a fused multi-scale feature F hi (where i is the number of feature layers), the formula is expressed as:

[0122] F hi =Flateen(F bi ),(i=2,3,4,5) (2)

[0123] Step 2.3: Obtain the fused multi-scale features F using concatenation.hi The fused features are then fed into the Transformer encoder, using the following formula:

[0124] F e =Concatenate(F hi ),(i=2,3,4,5) (3)

[0125] F e This represents the encoder features obtained from the hierarchical temporal fusion module and input to the Transformer.

[0126] The hierarchical temporal fusion module consists of two twin residual backbone feature extraction networks. The residual structure is one of the most popular backbone feature extraction structures currently available. Specifically, the feature extraction process is as follows:

[0127] First, perform a 7×7 convolution on the preceding and following time-phase images Image1 and Image2 respectively:

[0128]

[0129] Secondly, the features obtained in the previous step are processed through a batch normalization layer, a ReLU activation function layer, and a max pooling layer:

[0130]

[0131] Then, the obtained feature F′ conv1 and F′ conv2 Input into the residual cell to obtain Residual elements are typically divided into two types, referred to as basic structures and bottleneck structures, as shown in the figure. Figure 4 and Figure 5 As shown.

[0132] Depending on the number of residual units, residual backbone feature extraction networks can be categorized into several types, including ResNet18, ResNet34, ResNet50, and ResNet101. In this example, we use ResNet50 as the backbone feature extraction network, and the extracted hierarchical features can be represented as follows: in, and These represent the i-th layer features of the preceding and following time-phase images, respectively.

[0133] After extracting the i-th layer features from the preceding and following temporal images, the extracted preceding and following temporal layer features are concatenated according to the detailed steps described in step 2 of the technical solution, then flattened, and finally concatenated together to obtain the final feature F of the input encoder. e The process can be summarized as follows:

[0134]

[0135] Step 3: Refine the fused feature map to obtain a high-dimensional feature representation.

[0136] This invention employs an encoder-decoder-based network architecture. The encoder receives position embeddings and performs encoded predictions on bounding boxes and masks. Subsequently, a unified enhanced query selection module selects the top-ranked bounding boxes and masks from the encoder predictions to initialize content and anchor box queries. Specifically, positional encoding is added to the feature maps to preserve spatial information; multi-scale feature maps are flattened into a sequence and then processed through a series of Transformer encoding layers. These layers use self-attention mechanisms to further refine feature representations and deepen the model's understanding of image content. The top K features are selected from the last layer of the encoder to initialize the decoder's positional and content queries, passing refined features to the decoder to assist in accurate target localization and classification. The decoder accepts the encoder's output and uses cross-attention and mask self-attention mechanisms.

[0137] After obtaining the output feature F of the hierarchical time fusion module e The data is then fed into a Transformer-based encoder-decoder structure for further processing. This invention adopts the network structure of Mask DINO, a Transformer (DETEction Transformer, DETR) model for object detection with improved denoising anchor boxes. It adds a segmentation branch to the Transformer-based decoder to accurately output fine-grained edge representations of changing regions. Furthermore, this invention employs a query denoising strategy commonly used in object detection, inputting noisy labels into the encoder during training, effectively accelerating model convergence and improving the overall network performance. In this example, the encoder input is the feature F obtained in the previous step. e First, the feature is encoded and flattened to obtain the feature F. This process is represented as:

[0138] F = Flatten(F) e +Embedding) (7)

[0139] In the formula, E n-1 E′ represents the encoded feature of the (n-1)th layer. n This indicates that the encoded features of the (n-1)th layer after multi-head self-attention operation are related to E. n-1 The sum of E″ n E′ after passing through the feedforward network n With E′ n The sum of.

[0140] Then, through N encoder layers, each layer n includes a multi-head self-attention (MHA) network and a feedforward network (FFN), as follows:

[0141] E′ n =MHA(LayerNorm(E n-1 ))+E n-1 (8)

[0142] E″ n =FFN(LayerNorm(E′) n ))+E′ n (9)

[0143] Before the encoder outputs features into the decoder, the query content and anchor boxes are initialized first. This step is collectively referred to as query initialization:

[0144] Q = InitializeQueries() (10)

[0145] Using M decoder layers, each layer includes cross-attention (CA), self-attention (SA), and a feedforward network:

[0146] Q′ m =SA(CA(LayerNorm(Q) m-1 ),E"))+Q m-1 (11)

[0147] Q" m =FFN(LayerNorm(Q′) m ))+Q′ m (12)

[0148] Q m-1 This is the query output from the previous layer, where E" represents the encoded feature of the m-th layer, and Q′ m This represents the (m-1)th layer query Q that has undergone layer normalization, cross attention, and self attention operations in sequence. m-1 With Q m-1 The sum of, Q m This represents Q′ after layer normalization and feedforward network. m With Q′ m The sum of.

[0149] Step 4: Perform object-level change detection and instance-level segmentation processing on the high-dimensional features using a unified network, and obtain object-level bounding box representations and fine-grained boundary representations of the changing objects.

[0150] This invention adopts a dual-output mode of object-level change detection head and pixel-level segmentation output, simultaneously outputting the bounding box representation and fine boundary representation of the changing object. Therefore, the dual-branch output includes two parts: object-level change detection head and segmentation branch. The object-level change detection head is used to output the bounding box of the object, while the pixel-level segmentation branch is used to generate a fine-grained boundary representation (i.e., mask) for the changing object.

[0151] An object-level change detection head outputs bounding box representations of changed objects. The head employs a Look-forward-twice method derived from DINO, which corrects the predicted bounding box coordinates of adjacent previous layers by incorporating the bounding box coordinate information from the next layer. This strategy accelerates model convergence and improves the detection rate of small targets.

[0152] The split branch is used to generate a fine-grained boundary representation (i.e., a mask) for the modified object. First, it extracts 1 / 4 of the connectivity features F from the hierarchical temporal fusion module. b1 The feature map F is upsampled by 2 times and has a resolution of 1 / 8 compared to the decoder output. d The resulting pixel embeddings are then combined. Next, the resulting pixel embeddings are multiplied by the dot product of each content query embedding (Qc) in the decoder to perform the pixel embedding operation. This yields the final mask m, calculated as follows:

[0153]

[0154] In the formula, This represents the segmentation head, where Φ is a convolutional layer channel size mapped to the transformer's hidden dimension, and β is a parameter used to adjust F. b1 The convolution function Ψ for channel size is a simple function used to perform F d Execute the interpolation function with a 2x upsampling.

[0155] The object-level change detection head outputs the bounding box of the changed object and its corresponding category. In binary change detection, the output category is "change". This process is represented as:

[0156] Boxes = BoxRegression(Q″) m (14)

[0157] Classes=ClassClassification(Q″ m (15) Split branches to generate fine boundaries for the changing object, represented as:

[0158] Masks = MaskPrediction(Q″ m (16)

[0159] Among them, the high-dimensional feature Q" mThis represents the query features output by the m-th layer decoder. BoxRegression(), ClassClassification(), and MaskPrediction() represent bounding box regression, classification, and mask prediction operations, respectively. Boxes and Classes represent the bounding box coordinates and class of the changed object, respectively; Masks is the generated change boundary representation.

[0160] Step 5: Train the network using the obtained object-level bounding box representation and the fine boundary representation of the changing object, and output the change region detection results.

[0161] The network employs a hybrid loss function, which includes localization and classification losses for object-level change detection, and a masking loss for segmentation. The object-level change detection model is trained under this hybrid loss function optimization, outputting the final trained model and the predicted object-level change regions.

[0162] As a preferred embodiment, step 5 includes the following sub-steps:

[0163] Step 5.1: Calculate the localization loss for object-level change detection. Specifically, the localization loss L... box The minimum absolute value deviation loss and the generalized crossover ratio loss were adopted.

[0164] Step 5.2: Calculate the classification loss for object-level change detection. The classification loss is Focal loss, denoted as L. cls .

[0165] Step 5.3: Calculate the mask loss used for the segmentation task, which includes two parts: cross-entropy loss and dice loss.

[0166] Step 5.4: Linearly add the three loss functions calculated in Steps 5.1, 5.2, and 5.3 to obtain the total loss function L. hibird This can be expressed as a formula:

[0167] L hibird =λ cls L cls +λ L1 L L1 +λ GIOU L GUOU +λ ce L ce +λ dice L dice (17)

[0168] Where λ represents the weight settings among multiple loss functions, therefore, λ cls The weights λ represent the classification loss for object-level change detection.L1 λ represents the weights of the L1 loss for object detection. GIOU λ represents the weights of the generalized intersection-union loss for object detection. ce and λ dice These represent the weights of the cross-entropy loss and the dice loss, respectively.

[0169] Step 5.4: Using L from the aforementioned steps hibird As the loss function for training, the object-level change detection model is trained, and the final trained model and the object-level change region detection results are output.

[0170] This invention employs a hybrid loss function combining object-level change detection localization loss, classification loss, and mask loss for segmentation tasks. Through repeated parameter tuning and optimization, in this embodiment, λ is set... cls =4,λ L1 =5,λ GIOU =2,λ ce =5,λ dice =5.

[0171] Finally, the object-level change detection model is trained on the selected dataset, and the final trained model and object-level change region prediction results are output, obtaining the change box and refined boundary results for each change region.

[0172] Experiments were conducted on the publicly available LIM-CD dataset. Results on this dataset show that, based on the F1 score in a pixel-level accuracy evaluation system, the proposed method demonstrates improved change detection accuracy to varying degrees compared to existing methods based on CNN and Transformer, exhibiting superior performance in change detection tasks. Table 1 presents the experimental results of the proposed method and its comparative methods on LIM-CD; all values ​​are expressed as percentages (%).

[0173] Table 1: Experimental results of the method of the present invention and the comparative method on LIM-CD

[0174]

[0175]

[0176] The change detection results obtained by this invention are visualized, such as... Figure 6 As shown. Figure 6 In the image, from left to right: a is the previous phase image, b is the next phase image, c is the ground truth (label) image, d is the binary change detection result, e is the refined boundary of the change region (presented on the previous phase image), and f is the bounding box representation of the change region (presented on the next phase image).

[0177] The results above show that the present invention achieves the goals of accurately detecting change regions, accurately distinguishing each changing object within the change region, and outputting a fine boundary representation of the changing object, thus obtaining satisfactory change detection results. Furthermore, the present invention is the only Transformer-based object-level change detection method to date that simultaneously outputs the location of the change region and a fine boundary representation, and has significant value for further research.

[0178] An embodiment of the present invention provides a computer device including a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the device is running, the processor communicates with the memory via the bus, and the processor executes the machine-readable instructions to perform the steps of the method for detecting fine-grained object-level changes in any remote sensing image as described above.

[0179] Specifically, the aforementioned memory and processor can be general-purpose memory and processor, without any specific limitations. When the processor runs the computer program stored in the memory, it can execute the aforementioned method for detecting refined object-level changes in remote sensing images.

[0180] Those skilled in the art will understand that the structure of the computer device does not constitute a limitation on the computer device, and may include more or fewer components than shown in the figure, or combine some components, or split some components, or have different component arrangements.

[0181] In some embodiments, the computer device may further include a touchscreen for displaying a graphical user interface (e.g., an application launch screen) and receiving user actions on the graphical user interface (e.g., launching an application). Specifically, the touchscreen may include a display panel and a touch panel. The display panel may be configured as an LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), or similar type. The touch panel can collect user touch or non-touch operations on or near it and generate pre-set operation instructions, such as user actions using fingers, styluses, or any suitable object or accessory on or near the touch panel. Additionally, the touch panel may include a touch detection device and a touch controller. The touch detection device detects the user's touch position and posture, and detects the signals generated by the touch operation, transmitting the signals to the touch controller. The touch controller receives touch information from the touch detection device, converts it into information that the processor can process, sends it to the processor, and can also receive and execute commands from the processor. Furthermore, touch panels can be implemented using various types of sensors, including resistive, capacitive, infrared, and surface acoustic wave sensors, as well as any future technologies. Moreover, the touch panel can cover the display panel. Users can operate on or near the touch panel, which is covered by the graphical user interface displayed on the display panel. After detecting the operation on or near the touch panel, the touch panel transmits it to the processor to determine the user input. The processor then responds to the user input by providing corresponding visual output on the display panel. Additionally, the touch panel and display panel can be implemented as two separate components or integrated together.

[0182] Corresponding to the above application startup method, this embodiment of the invention also provides a storage medium storing a computer program, which, when run by a processor, executes the steps of the above-described method for detecting refined object-level changes in any remote sensing image.

[0183] The application launch device provided in this application embodiment can be specific hardware on the device or software or firmware installed on the device. The device provided in this application embodiment has the same implementation principle and technical effects as the foregoing method embodiments. For the sake of brevity, any parts not mentioned in the device embodiment can be referred to the corresponding content in the foregoing method embodiments. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can all be referred to the corresponding processes in the above method embodiments, and will not be repeated here.

[0184] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0185] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and there may be other division methods in actual implementation. Furthermore, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the coupling or direct coupling or communication connection shown or discussed may be through some communication interface, and the indirect coupling or communication connection of the apparatus or modules may be electrical, mechanical, or other forms.

[0186] The modules described as separate components may or may not be physically separate. Similarly, the components shown as modules may or may not be physical modules; they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0187] In addition, the functional modules in the embodiments provided in this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0188] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0189] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0190] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0191] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the protection scope of the claims of the present invention.

Claims

1. A method for detecting refined object-level changes in remote sensing images, characterized in that, The steps include the following: Step 1: Select a change detection dataset containing before-and-after temporal images and label datasets, and preprocess the change detection dataset; Step 2: Extract multi-scale hierarchical features from the preprocessed change detection dataset, and fuse the multi-scale hierarchical features to obtain a fused feature map; Step 3: Refine the fused feature map to obtain a high-dimensional feature representation; Step 4: Perform object-level change detection and instance-level segmentation on the high-dimensional features using a unified network, and obtain object-level bounding box representations and fine-grained boundary representations of the changed objects. Step 5: Train the network using the obtained object-level bounding box representation and the fine boundary representation of the changing object, and output the change region detection results; Step 2 includes: Step 21: Extract multi-scale hierarchical features from the dual-temporal images using a residual network, and fuse them using a connection operation: （1） in, and These represent the i-th layer features extracted from the preceding and following temporal images by the hierarchical temporal fusion module, respectively. ; Indicates a connection operation; Step 22, connect the hierarchical features Planarization into one-dimensional features : （2） in, Indicates the flattening operation; Step 23, for one-dimensional features Perform layered time fusion: (3) in, Features obtained after hierarchical temporal fusion; Step 3 includes: Step 31: Add positional encoding to the feature map; Step 32: The multi-scale feature maps are flattened into a sequence and processed through a series of Transformer encoding layers; Step 33: Select the top K features from the last layer of the encoder, perform position query and content query to initialize the decoder, and obtain the features to be processed; Step 33: Use cross-attention mechanism and mask self-attention mechanism to process the features to be processed and obtain high-dimensional feature representation; Step 4 includes: For high-dimensional features The bounding box representation of the changed object is output using the object-level change detection head: (14) (15) in, This represents the regression operation of the bounding box. Indicates the category of the changing objects. and These represent the bounding box coordinates and category of the changing object, respectively. For high-dimensional features Use split branches to generate fine-grained boundary representations for the modified objects: (16) in, Representing high-dimensional features, To represent the generated change boundary, () indicates a mask prediction operation.

2. The method for detecting refined object-level changes in remote sensing images according to claim 1, characterized in that, In step 1, the labeled dataset is a dataset of positive and negative samples of newly added construction land; the preprocessing of the change detection dataset is to enhance the change detection dataset by using a data augmentation method that randomly adds fog effects, randomly adds noise, randomly blurs, randomly grayscales, and limits contrast with adaptive histogram equalization.

3. The method for detecting refined object-level changes in remote sensing images according to claim 1, characterized in that, The extraction of multi-scale hierarchical features from dual-temporal images includes: Before and after time phase images , Perform a 7×7 convolution on each layer: (4) The features obtained from the convolution operation are then processed through a batch normalization layer, a ReLU activation function layer, and a max pooling layer. Features and The input is processed by the residual network to obtain the i-th layer features of the preceding and following temporal images.

4. The method for detecting refined object-level changes in remote sensing images according to any one of claims 1 to 3, characterized in that, Step 5 includes: Step 51: Calculate the positioning loss for object-level change detection, wherein the positioning loss includes minimum absolute value deviation loss and generalized intersection-union ratio loss; Step 52: Calculate the classification loss for object-level change detection using Focal loss; Step 53: Calculate the masking loss used for the segmentation task, which includes cross-entropy loss and dice loss; Step 54: Linearly add the loss functions of localization loss, classification loss and masking loss to obtain the total loss function; Step 55: Use the total loss function as the training loss function to train the object-level change detection model, and output the final trained model and the object-level change region detection results.

5. A device for detecting refined object-level changes in remote sensing images, used to implement the method for detecting refined object-level changes in remote sensing images as described in any one of claims 1 to 4, characterized in that, include: The dataset acquisition module is used to select a change detection dataset containing before-and-after temporal images and label datasets, and to preprocess the change detection dataset. The feature fusion module is used to extract multi-scale hierarchical features from the preprocessed change detection dataset and fuse the multi-scale hierarchical features to obtain a fused feature map. The feature extraction module is used to refine the fused feature map and obtain a high-dimensional feature representation. The boundary representation module is used to perform object-level change detection and instance-level segmentation on high-dimensional features using a unified network, while obtaining object-level bounding box representations and fine-grained boundary representations of changing objects. The network training module is used to train the network using a hybrid loss function based on the obtained object-level bounding box representation and the fine boundary representation of the changing object, and outputs the change region detection results.

6. A computer device, characterized in that, The device includes a processor, a memory, and a bus. The memory stores machine-readable instructions that the processor can execute. When the computer device is running, the processor communicates with the memory via the bus, and the processor executes the machine-readable instructions to perform the steps of the remote sensing image fine-grained object-level change detection method as described in any one of claims 1 to 4.

7. A storage medium, characterized in that, The storage medium stores a computer program that, when executed by a processor, performs the steps of the method for detecting refined object-level changes in remote sensing images as described in any one of claims 1 to 4.

Images