Remote sensing image change detection method based on full-scale gating message propagation framework
By employing a full-scale gated message propagation framework and utilizing twin networks and dense fusion modules, the accuracy of remote sensing image change detection was improved, solving the challenges of detecting small-scale changing objects and edges, and achieving higher detection accuracy and more complete small target capture.
Patent Information
- Application Number
- CN202310103145.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-02-13
- Publication Date
- 2026-02-27
- Estimated Expiration
- 2043-02-13
AI Technical Summary
Existing technologies struggle to accurately capture small-scale changes and the edges of changing targets in high-resolution remote sensing image change detection, especially due to interference from complex surface environments and multi-temporal images, resulting in insufficient detection accuracy.
A full-scale gated message propagation framework is adopted. Multi-scale features are extracted through a Siamese network structure, and key embedding representations are generated by a collaborative estimation module as gate switches to achieve full-scale message transmission. Shallow pixel information and deep semantic information are fused through a dense fusion module to improve detection accuracy.
It significantly improves the detection accuracy at the edges of objects with small changes and changing targets, reduces false detection pixels, and achieves more complete small target detection and more accurate edge segmentation.
Smart Images
Figure CN116310797B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application belongs to the technical field of digital image processing, and particularly relates to a remote sensing image change detection method based on a full-scale gated message passing framework, which can improve the detection accuracy of small change objects and change target edges. BACKGROUND
[0002] In recent years, with the improvement of earth observation technology with the characteristics of "high precision, all directions, short cycle" and the development of optical sensors, high-resolution remote sensing images have become a new carrier for remote sensing detection targets, and change detection based on high-resolution remote sensing images has been applied to many research fields such as map making, disaster assessment and urban planning.
[0003] The accurate geometric information and position information contained in high-resolution remote sensing images can clearly reflect the information of various scale ground objects in the observed area, making it possible to capture small-scale ground object changes. However, the complexity of high-resolution imaging information brought by complex surface environment also brings new challenges to high-resolution image change detection, which are embodied in the following aspects: first, the diversity of ground object expression, mainly manifested in the multiple scales, various shapes and random positions of ground objects; second, the heterogeneity of change ground object expression, including multiple interferences caused by seasonal replacement, shadow coverage and other interference factors; third, the sensitivity of multi-temporal high-resolution images to time period, which greatly increases the difficulty of controlling false change information. Therefore, how to face the task of double-temporal high-resolution change detection, while ensuring the correct prediction of the position of change ground objects, and as accurately as possible capturing the boundaries of change targets and small-scale change ground objects, is still a key problem to be solved in the field.
[0004] In recent research, thanks to a large number of visual interpretation of labeled sample work and the improvement of hardware computing power, deep learning-based methods have become a research hotspot in the field of high-resolution remote sensing image change detection due to their powerful nonlinear data fitting ability and automatic processing ability, and many effective solutions have been proposed. However, due to insufficient input cutting or feature utilization, the change information capturing ability of these networks is often limited, which affects the detection accuracy of small change objects and change target edges. SUMMARY
[0005] The present application is to solve the above technical problems existing in the prior art, and provides a remote sensing image change detection method based on a full-scale gated message passing framework, which can improve the detection accuracy of small change objects and change target edges.
[0006] The technical solution of the present application is: a remote sensing image change detection method based on a full-scale gated message passing framework, which is performed in the following steps in turn:
[0007] Step 1. Preprocess the T1 time phase high-resolution remote sensing image and the T2 time phase high-resolution remote sensing image of the test set respectively to obtain the T1 time phase high-resolution remote sensing image test set and the T2 time phase high-resolution remote sensing image test set
[0008] Step 1.1. Denote the T1 time phase high-resolution remote sensing image data block of the test set as X H1 H1，i i=1, …, M), and denote the T2 time phase high-resolution remote sensing image data block of the test set as X H2 H2，i i=1, …, M), where M is the number of data blocks of each type;
[0009] Step 1.2. Standardize the high-resolution remote sensing image data blocks X H1 and X H2 using formula (1) to obtain
[0010]
[0011] Step 1.3. Take the data block as the T1 time phase high-resolution remote sensing image test set list, that is, the T1 time phase high-resolution remote sensing image test set Take the data block as the T2 time phase high-resolution remote sensing image test set list, that is, the T2 time phase high-resolution remote sensing image test set
[0012] Step 2. Extract the i-th sub-block in the T1 time phase high-resolution remote sensing image detection set and the T2 time phase high-resolution remote sensing image detection set respectively, and input them into the trained network structure in the form of to obtain the final detection result;
[0013] The trained network structure is constructed in the following steps:
[0014] Step 2.1. Preprocess the T1 time phase high-resolution remote sensing image, the T2 time phase high-resolution remote sensing image and the corresponding label image in the training set respectively to obtain the T1 time phase high-resolution remote sensing image training set the T2 time phase high-resolution remote sensing image training set and the label training set
[0015] Step 2.1.1. Denote the T1 time phase high-resolution remote sensing image data block of the training set as X H3 H3，i i = 1, …, M), the T2 time phase high-resolution remote sensing image data block of the training set is X H4 = {x H4，i i = 1, …, M), the label data block of the training set is X L = {x L，i i = 1, …, M), wherein M is the number of data blocks of each class;
[0016] Step 2.1.2, the data blocks X H3 , X H4 and X L are standardized by formula (2) to obtain
[0017]
[0018] Step 2.1.3, each pixel point in the high-resolution remote sensing image data block and the label data block is flipped up and down respectively to obtain the high-resolution remote sensing image data block flipped up and down and the label data block flipped up and down
[0019] Step 2.1.4, each pixel point in the high-resolution remote sensing image data block and the label data block is flipped left and right respectively to obtain the high-resolution remote sensing image data block flipped left and right and the label data block flipped left and right
[0020] Step 2.1.5, the high-resolution remote sensing image data block and the label data block are rotated clockwise by 90 degrees a times, a = 1, 2, 3, to obtain the rotated high-resolution remote sensing image data block and the rotated label data block
[0021] Step 2.1.6, the data block is combined with the data block and to constitute a new T1 time phase high-resolution remote sensing image training set list, that is, the T1 time phase high-resolution remote sensing image training set The data block is combined with the data block and to constitute a new T2 time phase high-resolution remote sensing image training set list, that is, the T1 time phase high-resolution remote sensing image training set The label data block is combined with the label data block To form a new list of labeled training sets, i.e., the labeled training set
[0022] Step 2.2 Set the total number of network iterations, epoch = T;
[0023] Step 2.3 Set epoch = 1;
[0024] Step 2.4 Convert the training set samples into... The input is given in the form of binary cross-entropy, and optimized using Adam.
[0025] Step 2.4.1 Utilize the pre-trained ResNet18 network as an encoder to process the high-resolution remote sensing image samples x. T1,i ,x T2,i Extracting multi-scale features, let the extracted T1 phase multi-scale features be denoted as follows: The extracted T2 phase multi-scale features are as follows:
[0026] Step 2.4.2: Combine the same-level feature maps from different time phases. and The data is input in pairs into the collaborative estimation module m, and key embedding representations are generated for them. and This serves as the switch for the gate control:
[0027] Step 2.4.2.1 Utilize a transform layer with convolution and activation operations to... and Dimensionality reduction was performed separately, and the dimensionality-reduced values were... and It has 64 channels;
[0028] Step 2.4.2.2 is and Randomly generate a set of compact bases As a channel descriptor, and calculate the corresponding spatial descriptor according to formula (3).
[0029]
[0030] Step 2.4.2.3 Update according to formula (4) and
[0031]
[0032] Step 2.4.2.4 will splice the... and Input to the linear layer to extract common features γ in it m ;
[0033] Step 2.4.2.5 fuses the channel descriptor and the spatial descriptor for each feature map respectively as its key embedding expression according to formula (5);
[0034]
[0035] Step 2.4.3 takes and as inputs of the full-scale gating mechanism to control the message sending of each feature map F m according to formula (6), wherein n = 1, 2, m = 1, 2, 3, 4;
[0036]
[0037] Step 2.4.4 inputs F m to the dense fusion module for decoding, and the decoded feature map is denoted as
[0038] Step 2.4.4.1 concatenates F m , and the concatenated feature is denoted as
[0039] Step 2.4.4.2 obtains
[0040]
[0041] wherein i = 1, 2, 3, 4, Mapping() is a mapping layer with convolution, batch normalization and activation operations, is the output of the mapping layer, concat() is a concatenation layer, is the output of the concatenation layer;
[0042] Step 2.4.5 inputs the output of the previous step to a mapping layer with convolution, batch normalization and activation operations, and the output of the layer is denoted as F map ;
[0043] Step 2.4.6 inputs the output of the previous step F map to a scaling layer with bilinear interpolation, batch normalization, bilinear interpolation and activation operations, and the output of the layer is denoted as F re ;
[0044] Step 2.4.7 inputs the output of the previous step F re to a classification layer with convolution operations, and the output of the layer is denoted as Pred;
[0045] Step 2.4.8 calculates the difference between Pred and Y according to the binary cross-entropy loss function shown in formula (8);
[0046]
[0047] Here N is the number of pixels in Y, Pred i,j is the coordinate value at the position in the prediction change map, Y i,j is the value of the true value map at the same position;
[0048] Step 2.4.9 determines whether all sub-blocks in the training set have been processed, yes, go to step 2.4.10, no, return to step 2.4.1;
[0049] Step 2.4.10 epoch <- epoch + 1, if epoch > T, the trained network parameters are obtained, otherwise, the network parameters are optimized using Adam, and step 2.4.1 is entered.
[0050] The present application consists of the following four parts: 1) using a weight-shared twin network structure as an encoder to extract multi-scale features; 2) using a collaborative estimation module to generate a set of common compact bases for the same level feature maps across time phases, and then inferring the key embedding expressions of each time phase based on the set of bases as the gating switch; 3) using a full-scale gating mechanism as the message passing mode between the double time phase images, which is different from the previous modeling method only considering the same level feature maps, so that information is transmitted in both the full-scale same phase and the cross-time phase; 4) using a new step-by-step connection to a dense fusion module as a decoder to harmonize the shallow pixel information and the deep semantic information in the same feature map, and finally combining the relevant recovery spatial size operation to predict the change map.
[0051] The present application can realize double-path message passing in the full-scale same phase and the cross-time phase under the full-scale gating mechanism, and determine whether each position feature is sent and the amount of transmission through the gating switch, propagate the real change information while filtering out invalid pseudo change information. The gating switch is obtained by the collaborative estimation module, which can generate key embedding expressions for the same level feature maps on a set of compact common bases, making the difference features between the double time phases more significant. Finally, the dense fusion module merges the long and short skip connections to perform step-by-step multiple fusion of features of each scale, so that the shallow pixel information and the deep semantic information are harmonized in the same feature map, enhancing the feature expression ability of the network and improving the detection accuracy of small change objects and change target edges. BRIEF DESCRIPTION OF DRAWINGS
[0052] Figure 1 is the network structure and training step schematic diagram of the embodiment of the present application.
[0053] Figure 2is a schematic diagram of a cooperative estimation module of an embodiment of the present application.
[0054] Figure 3 is a schematic diagram of a full-scale gating mechanism of an embodiment of the present application.
[0055] Figure 4 is a comparison chart of the visual detection results of the method of the present application and prior art and the number of identified error pixels in the chart. DETAILED DESCRIPTION
[0056] A remote sensing image change detection method based on a full-scale gating message propagation framework of the present application is performed in turn according to the following steps:
[0057] Step 1. Preprocess the T1 time phase high-resolution remote sensing image test set and the T2 time phase high-resolution remote sensing image test set of the test set respectively, to obtain the T1 time phase high-resolution remote sensing image test set and the T2 time phase high-resolution remote sensing image test set.
[0058] Step 1.1. Denote the data block of the T1 time phase high-resolution remote sensing image of the test set as X H1 ={x H1,i |i=1,…,M}, and denote the data block of the T2 time phase high-resolution remote sensing image of the test set as X H2 ={x H2,i |i=1,…,M}, where M is the number of each type of data block.
[0059] Step 1.2. Standardize the high-resolution remote sensing image data blocks X H1 and X H2 using formula (1) to obtain
[0060]
[0061] Step 1.3. Take the data block as the T1 time phase high-resolution remote sensing image test set list, i.e., the T1 time phase high-resolution remote sensing image test set Take the data block as the T2 time phase high-resolution remote sensing image test set list, i.e., the T2 time phase high-resolution remote sensing image test set
[0062] Step 2. Extract the i-th sub-block in the T1 time phase high-resolution remote sensing image detection set and the T2 time phase high-resolution remote sensing image detection set respectively, and input them into the trained network structure in the form of to obtain the final detection result.
[0063] The trained network structure is constructed sequentially according to the following steps:
[0064] Step 2.1 Preprocess the high-resolution remote sensing images at time T1 and time T2 in the training set, along with their corresponding label maps, to obtain the high-resolution remote sensing image training set at time T1. T2 temporal high-resolution remote sensing image training set and tag training set
[0065] Step 2.1.1 Denote the T1 time-phase high-resolution remote sensing image data block of the training set as X H3 ={x H3,i |i=1,…,M}, the T2 time-phase high-resolution remote sensing image data block of the training set is X H4 ={x H4,i The labeled data block of the training set is X, where |i=1,…,M}. L ={x L,i |i=1,…,M}, where M is the number of data blocks in each class;
[0066] Step 2.1.2 Use formula (2) to divide data block X H3 X H4 and X L Standardization process is performed to obtain
[0067]
[0068]
[0069] Step 2.1.3 Separate the high-resolution remote sensing image data blocks and tag data blocks Each pixel in the image is flipped vertically to obtain a high-resolution remote sensing image data block that has been flipped vertically. and the label data blocks that are flipped up and down
[0070] Step 2.1.4 Separate the high-resolution remote sensing image data blocks and tag data blocks Each pixel in the image is flipped horizontally to obtain a horizontally flipped high-resolution remote sensing image data block. and horizontally flipped label data blocks
[0071] Step 2.1.5 Separate the high-resolution remote sensing image data blocks and tag data blocks Rotate clockwise by 90 degrees a times, where a = 1, 2, 3, to obtain the rotated high-resolution remote sensing image data block. and rotating label data blocks
[0072] Step 2.1.6 combines the data blocks to form a new T1 time phase high-resolution remote sensing image training set list, that is, the T1 time phase high-resolution remote sensing image training set to form a new T2 time phase high-resolution remote sensing image training set list, that is, the T1 time phase high-resolution remote sensing image training set to form a new label training set list, that is, the label training set
[0073] Step 2.2 sets the total number of network iterations epoch = T;
[0074] Step 2.3 sets epoch = 1;
[0075] Step 2.4 inputs the training set samples into the network in the form of Figure 1 , takes binary cross entropy as the loss function, and uses Adam for optimization:
[0076] Step 2.4.1 uses the encapsulated pre-trained Resnet18 network as an encoder to extract multi-scale features from the high-resolution remote sensing image samples x T1,i ,x T2,i , and the extracted T1 time phase multi-scale features are respectively , and the extracted T2 time phase multi-scale features are respectively
[0077] Step 2.4.2 inputs the feature maps of the same level of different time phases and into the collaborative estimation module m in pairs, and generates key embedding expressions and for them, which are used as the switch of gating, as shown in Figure 2
[0078] Step 2.4.2.1 uses a transformation layer with convolution and activation operations to reduce the dimensions of and , respectively, and the reduced and have 64 channels;
[0079] Step 2.4.2.2 is With A set of compact bases is randomly generated As channel descriptors, and the spatial descriptors corresponding to them are calculated according to formula (3)
[0080]
[0081] Step 2.4.2.3 updates With
[0082]
[0083] Step 2.4.2.4 concatenates With Input to the linear layer to extract the common features γ m therein;
[0084] Step 2.4.2.5 fuses the channel descriptor and the spatial descriptor for each feature map respectively according to formula (5) as its key embedding expression;
[0085]
[0086] Step 2.4.3 is With As the input of the full-scale gating mechanism, according to formula (6) to control the message sending of each feature map F m , and the full-scale gating mechanism is as shown in Figure 3 ;
[0087]
[0088] Step 2.4.4 inputs F m to the dense fusion module for decoding, and the decoded feature map is denoted as
[0089] Step 2.4.4.1 concatenates F m (m = 1, 2, 3, 4) to denote the concatenated features as
[0090] Step 2.4.4.2 obtains
[0091]
[0092] Where i = 1, 2, 3, 4, Mapping() is a mapping layer with convolution, batch normalization and activation operations, is the output of the mapping layer, and concat() is a concatenation layer, This is the output of the splicing layer;
[0093] Step 2.4.5 will output the data from the previous step. The input is fed into a mapping layer that performs convolution, batch normalization, and activation operations, and the output of this layer is denoted as F. map ;
[0094] Step 2.4.6 convert the F output from the previous step map The input is fed into a scaling layer that performs bilinear interpolation, batch normalization, bilinear interpolation, and activation operations. Let the output of this layer be F. re ;
[0095] Step 2.4.7 convert the F output from the previous step re The input is fed into a classification layer with convolutional operations, and the output of this layer is denoted as Pred.
[0096] Step 2.4.8 Calculate the difference between Pred and Y according to the binary cross-entropy loss function shown in formula (8);
[0097]
[0098] Here, N is the number of pixels in Y, Pred i,j It is the coordinate value of the location in the predicted change graph, Y. i,j These are the values of the truth map at the same location;
[0099] Step 2.4.9 Determine whether all sub-blocks in the training set have been processed. If yes, proceed to step 2.4.10; otherwise, return to step 2.4.1.
[0100] Step 2.4.10 epoch←epoch+1, if epoch>T, then the trained network parameters are obtained; otherwise, continue to optimize the network parameters using Adam and proceed to step 2.4.1.
[0101] The embodiments of the present invention are compared with the existing SDACD method and RDP-Net method for processing some CDD datasets (including the visualization of detection result images and the statistics of the number of incorrectly identified pixels in the images). The results are as follows: Figure 4 As shown.
[0102] The comparison results show that:
[0103] The detection results of this invention are significantly superior to those of the SDACD and RDP-Net methods. Specifically, the advantages of this invention are:
[0104] 1. More accurate detection of edges: The detection edge of the present application has fewer false detection pixels, especially in the subjective detection results of Examples 1, 2 and 3. From the difference map between the detection results and the true value map, it can be observed that the comparison method has more false detection pixels at the detection edge. Specifically, the detection accuracy of Example 1 is 131% higher than the second place RDP-Net method, the detection accuracy of Example 2 is 132% higher than the second place RDP-Net method, and the detection accuracy of Example 3 is 272% higher than the second place RDP-Net method. The method proposed in the present application has the highest detection accuracy and the best detection effect.
[0105] 2. More complete small target detection: Example 4 more clearly shows the advantages of the method proposed in the present application in small target detection. The SDACD method and the RDP-Net method perform poorly in Example 4. Firstly, they do not correctly detect the shape of the changing target. Secondly, the edges of the detection are not continuous enough. However, the present application achieves the above two points well, that is, it completely captures the changes of small targets and correctly segments their edges. The detection accuracy is 131% higher than the second place SDACD method, achieving the purpose of high-precision detection.
Claims
1. A remote sensing image change detection method based on a full-scale gated message passing framework, characterized in that The following steps are performed in sequence: Step 1. Preprocess the T1 time phase high-resolution remote sensing image and the T2 time phase high-resolution remote sensing image of the test set respectively to obtain the T1 time phase high-resolution remote sensing image test set and the T2 time phase high-resolution remote sensing image test set : Step 1.1 Let the T1 time phase high-resolution remote sensing image data blocks of the test set be denoted as , and the T2 time phase high-resolution remote sensing image data blocks of the test set be denoted as , where M is the number of data blocks of each type. Step 1.2 Standardize the high-resolution remote sensing image data block using formula (1) to obtain and , ; (1) Step 1.3 Block data as a list of T1 high-resolution remote sensing image test sets, i.e. T1 high-resolution remote sensing image test sets ; Block data as a list of T2 high-resolution remote sensing image test sets, i.e. T2 high-resolution remote sensing image test sets ; Step 2. Extract the T1 phase high-resolution remote sensing image detection set respectively and the first sub-block in the T2 phase high-resolution remote sensing image detection set in the form of input into the trained network structure to obtain the final detection result; The trained network structure is constructed in sequence according to the following steps: Step 2.1, respectively, the training set T1 time phase high-resolution remote sensing image, T2 time phase high-resolution remote sensing image and corresponding label image are preprocessed to obtain T1 time phase high-resolution remote sensing image training set , T2 time phase high-resolution remote sensing image training set and label training set : Step 2.1.
1. Let the T1 time phase high-resolution remote sensing image data block of the training set be denoted as , the T2 time phase high-resolution remote sensing image data block of the training set be denoted as , and the label data block of the training set be denoted as , where M is the number of each type of data block. Step 2.1.2 Standardize the data blocks using formula (2) , and to obtain , , ; (2) Step 2.1.3 Flip up and down each pixel point in the high-resolution remote sensing image data block , and the label data block to obtain the up and down flipped high-resolution remote sensing image data block , and the up and down flipped label data block ; Step 2.1.4 horizontally flips each pixel point in the high-resolution remote sensing image data block , and the label data block to obtain the horizontally flipped high-resolution remote sensing image data block , and the horizontally flipped label data block , respectively. Step 2.1.5 Rotate the high-resolution remote sensing image data block , and the label data block clockwise by 90 degrees a times, a = 1, 2, 3, to obtain the rotated high-resolution remote sensing image data block , and the rotated label data block ; Step 2.1.6 combines data blocks , , and to form a new list of T1 time phase high resolution remote sensing image training sets, namely T1 time phase high resolution remote sensing image training set ; Step 2.1.7 combines data blocks , , and to form a new list of T2 time phase high resolution remote sensing image training sets, namely T2 time phase high resolution remote sensing image training set ; , , and to form a new list of label training sets, namely label training set . Step 2.2 Set total number of network iteration ; Step 2.3 order ; Step 2.4 The training set samples are input into the network in the form of binary cross-entropy as the loss function, and Adam is used for optimization: Step 2.4.1 uses the packaged pre-trained Resnet18 network as an encoder to respectively extract high-resolution remote sensing image samples , Extract multi-scale features, and extract T1 phase multi-scale features respectively Extracted T2 phase multi-scale features are , ; Step 2.4.2: Concatenate features of the same level from different time steps with are inputted into the co-estimation module and generate key embedding representations for them with as a switch for gating: Step 2.4.2.1 Utilize transform layers with convolution and activation operations with respectively; and with having 64 channels; Step 2.4.2.2 is With generate a set of compact bases , , as channel descriptors, and compute the spatial descriptors corresponding to them according to equation (3) , ; (3) Step 2.4.2.3 Update according to formula (4) , ; (4) Step 2.4.2.4 Concatenate the outputs from the previous steps , Input to the linear layer to extract common features among them ; Step 2.4.2.5: According to formula (5), the feature map fusion channel descriptor and the spatial descriptor are respectively taken as the key embedding expression thereof; (5) Step 2.4.3 will and As the input to the full-scale gating mechanism, each feature map is controlled according to formula (6). The message sending status, the ; (6) Step 2.4.4.4 The input is decoded by the dense fusion module, and the decoded feature map is denoted as : Step 2.4.4.1 Splicing , the spliced feature is denoted as ; Step 2.4.4.2 Obtain according to formula (7) ; (7) In the formula , is a mapping layer with convolution, batch normalization and activation operations, is the output of the mapping layer, is a concatenation layer, is the output of the concatenation layer; Step 2.4.5 The output of the previous step is input to a mapping layer with convolution, batch normalization, and activation operations, denoted as ; Step 2.4.6 The output of the previous step is input to a scaling layer with bilinear interpolation, batch normalization, bilinear interpolation, and activation operations, denoted as ; Step 2.4.7 The output of the previous step is input to a classification layer with convolutional operations, denoted as ; Step 2.4.
8. Calculate the binary cross-entropy loss function as shown in Equation (8) differences between differences between (8) Here is the number of pixels in the image, is the coordinate value at the position in the prediction change map, is the value of the ground truth map at the same position; Step 2.4.9: Determine whether all sub-blocks in the training set are processed, yes, go to step 2.4.10, no, return to step 2.4.1; Step 2.4.10 If , the trained network parameters are obtained, otherwise, the network parameters are optimized using Adam, and the process goes to step 2.4.1.