A change detection method based on semantic alignment and feature enhancement

Abstract

The application provides a change detection method based on semantic alignment and feature enhancement, which solves the problem of poor edge detection accuracy and the like, and comprises the following steps: S1: constructing a three-branch feature extractor with a ResNet34 network structure in the encoding stage, wherein two time phase feature extraction branches share parameters F1 and F2, and the fusion feature branch parameter F3 / F is independent; S2: fusing and scheduling the three-branch features through a semantic flow alignment module FAM, and marking the low-dimensional pooled features from the same branch as F3 and the fusion results of the other two branches as F; S3: the fusion features F are used as remote information to perform information enhancement through an attention feature enhancement module FEM and are superimposed with the same dimension information in the decoding stage; and S4: the decoding stage restores the resolution layer by layer to be consistent with the original image through a transpose convolution group TCG combined with an inverted residual, and finally outputs a pixel-by-pixel change recognition result. The application has the advantages of high edge detection accuracy and the like.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of remote sensing change detection, and particularly relates to a change detection method based on semantic alignment and feature enhancement. BACKGROUND

[0002] Remote sensing change detection is a process of identifying changes on the earth's surface by jointly processing dual-temporal (or multi-temporal) images acquired in the same geographic area. It is widely used in many fields, such as land use / cover, disaster monitoring and assessment, cryosphere monitoring, ecological environment protection, etc. Traditional pixel-oriented change detection methods, such as Change Vector Analysis (CVA), Principal Component Analysis (PCA), and Iteratively Reweighted Multivariate Alteration Detection (IRMAD), ignore spatial context relationships and are prone to produce more false and noise points in small areas, which are not sufficient to represent the key change information of the image. Object-oriented change detection methods are limited by image segmentation techniques and usually rely on manually adjusted low-level visual image features, which often work well only in specific scenarios. In recent years, deep learning technology has realized the autonomous learning of image features and generally achieved higher accuracy than traditional methods, so it has been widely studied and applied in the field of remote sensing. At present, the application of deep learning technology in the field of remote sensing change detection has become a hot spot and has made significant progress. Although the twin network model using attention mechanism has better performance than traditional methods, there is still a phenomenon of semantic misalignment between dual-temporal images, and there is a pixel shift between the same ground objects in different temporal phases. Researchers often use a twin network structure modified from a semantic segmentation model to extract features in the encoding stage, which shares parameters but does not exchange information. The pixel-level shift phenomenon easily leads to inaccurate edge extraction. In addition, the attention mechanism is more used to strengthen feature extraction, and the way of channel stacking of dual-temporal feature maps when using skip connection cannot effectively fuse information and highlight change features. In the change detection scenario, it is necessary to realize dual-temporal feature fusion, enhancement, and more effective long-range information transmission.

[0003] To address the shortcomings of existing technologies, researchers have conducted long-term research and proposed various solutions. For example, Chinese patent literature discloses a remote sensing image change detection method based on spatiotemporal attention [202111366545.0], which includes: Step 1: preparing remote sensing image pairs; Step 2: performing data preprocessing on the prepared remote sensing image pairs and the corresponding mask images of the change regions; Step 3: inputting the remote sensing image pairs into a Siamese convolutional neural network to obtain the first feature map and the second feature map corresponding to the image pairs; Step 4: concatenating the first feature map and the second feature map in the time dimension to obtain feature map X, which is then input into the spatiotemporal attention module; Step 5: using an improved contrastive loss function to process the feature map of the image pairs output by the Siamese neural network from the tensor Z output by the spatiotemporal attention module.

[0004] The above-mentioned scheme has solved the technical problem of pseudo-change detection to a certain extent, but it still has many shortcomings, such as poor accuracy in edge detection. Summary of the Invention

[0005] The purpose of this invention is to address the above-mentioned problems by providing a well-designed change detection method based on semantic alignment and feature enhancement that achieves high edge detection accuracy.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: a change detection method based on semantic alignment and feature enhancement, comprising the following steps:

[0007] S1: In the encoding stage, a three-branch feature extractor is constructed using the ResNet34 network structure, in which the parameters F1 and F2 of the two temporal feature extraction branches are shared, and the parameters F3 and F of the fusion feature branch are independent;

[0008] S2: The three-branch features are fused and scheduled by the semantic flow alignment module FAM. The low-dimensional pooled features from the same branch are labeled as F3, and the fusion result with the other two branches is labeled as F.

[0009] S3: The fusion feature F, as remote information, is enhanced by the attention feature enhancement module FEM and superimposed with the information of the same dimension in the decoding stage;

[0010] S4: In the decoding stage, the resolution is restored layer by layer to match the original resolution by combining the transposed convolutional group TCG of the inverted residual. Figure 1 Finally, the output is the pixel-by-pixel change recognition result. ResNet is a commonly used feature extractor. Traditional deep networks may experience accuracy saturation or even regression as the network deepens. The addition of residual modules changes the local structure of the network to change the learning target. Instead of fitting the target mapping, it fits the residual mapping. The module can effectively alleviate the gradient vanishing phenomenon in the network during the learning process.

[0011] In the change detection method based on semantic alignment and feature enhancement, the fusion feature branch in step S1 is initially obtained by a double-branch feature value superposition manner.

[0012] In the change detection method based on semantic alignment and feature enhancement, the semantic flow alignment module FAM in step S2 includes the following steps:

[0013] S21: The two branch extraction features F1 and F2 are channel spliced and subjected to channel-invariant convolution operation to realize preliminary fusion, and then channel spliced and convolved with F3 to obtain a position offset map;

[0014] S22: The offset map value is a preliminary information fusion result corresponding to the index of the fusion branch position, and the index information is scheduled to realize semantic alignment of the feature map, and the scheduling result is value superposed with the F3 feature subjected to twice convolution operation, realizing three-branch information fusion, and the fusion process is as follows:

[0015] F=Warp[F cov,Gelu [F cat,,conv (F1,F2),F cov (F3)],FF cat,conv (F1,F2)]+F cov (F3)。

[0016] The semantic flow alignment module FAM realizes semantic precise alignment of double-time-phase features to complete feature fusion in the encoding stage.

[0017] In the change detection method based on semantic alignment and feature enhancement, the activation function in step S2 adopts a Gaussian error linear unit GELU.

[0018] In the change detection method based on semantic alignment and feature enhancement, the module inputs F1, F2 and F3 in step S2 are time phase 1 feature map, time phase 2 feature map and last level fusion feature down-sampling feature map respectively, and the double-time-phase features F1 and F2 are scheduled with F3 as a guide, and the scales are CxHxW, and the three are fused into a feature F.

[0019] In the change detection method based on semantic alignment and feature enhancement, the attention feature enhancement module FEM in step 3 first aggregates spatial information in the channel strengthening branch, and first integrates channel information in the spatial strengthening branch, and then calculates the weight after feature fusion, and outputs the channel enhanced feature CF and the spatial enhanced feature S, and superimposes the input feature F to obtain the sum EF. The superposition of the input feature makes the attention feature enhancement module FEM tend to learn the feature residual.

[0020] In the change detection method based on semantic alignment and feature enhancement, the attention feature enhancement module FEM in step 3 realizes feature enhancement in the channel dimension through channel attention, and the specific steps are as follows:

[0021] S31: Two groups of feature maps are obtained through 1*1 convolution and 3*3 convolution, wherein V1 completely compresses the channel, and V2 maintains the original resolution;

[0022] S32: V1 changes the shape to HW1*1*1, and again obtains the weight C*1*1 through 1*1 convolution and an activation function, V2 changes the matrix dimension and shape to HW*C, and through 1*1 convolution, the dimension is reduced to 1*C to realize the aggregation of spatial information, the HW position information is included in the same pixel, and then the weight is filtered through an activation function;

[0023] S33: The weights obtained in two different ways perform matrix multiplication, and then an activation function is used to obtain the channel feature weight CW(C*C), and finally the input feature is multiplied to obtain the channel enhanced feature CF, and the channel enhanced feature operation formula is:

[0024] CF=F cov (F)×F Gelu [F cov,Gelu (F reshape (V1))×F cov,Gelu (F reshape (V2))]。

[0025] In the change detection method based on semantic alignment and feature enhancement, the attention feature enhancement module FEM in step 3 realizes feature enhancement in the channel dimension through channel attention, and the specific steps are as follows:

[0026] S34: V3 changes the feature matrix shape to C*HW, and then through 1*1 convolution, the channel is mapped and integrated, and an activation function is used for enhancement, V4 first uses 1*1 convolution to realize neighborhood information interaction and channel compression, obtains multi-channel integrated features, and the shape is 1*H*W, and then shape remodeling (HW*1*1) and activation enhancement are performed;

[0027] S35: The two groups of weights perform matrix multiplication, and then an activation function is used to obtain the spatial feature weight SW(HW*HW), which is multiplied with the input feature and then reshaped to output the spatial enhanced feature SF, and the spatial enhanced feature operation formula is:

[0028] SF=F cov (F)×F Gelu [F cov,Gelu (V3)×F Gelu (F reshape (V4))]。

[0029] In the change detection method based on semantic alignment and feature enhancement, the transpose convolution group TCG combined with the inverted residual in step 4 comprises the following steps:

[0030] S41: The feature accumulation is used to sum the values of the high-dimensional deconvolution feature DF and the feature enhancement module output feature EF in the same dimension, so as to realize the remote information transmission of the encoding process and the decoding process;

[0031] S42: The learning ability of the inverted residual enhancement model is used;

[0032] S43: The transpose convolution is used to realize the upsampling of the feature map to the low-dimensional feature LDF.

[0033] In the change detection method based on semantic alignment and feature enhancement, the inverted residual in step S42 increases the channel number to 3 times the input channel number in 1*1, and then reduces the dimension after feature extraction. The inverted residual module can expand the feature information and optimize the residual calculation ability of the network structure on the one hand, and can promote the model to learn the residual between the feature value and the true value, and avoid the gradient disappearance of the model.

[0034] Compared with the prior art, the advantages of the present application are that the three-branch feature is fused and the semantic error is eliminated by the semantic flow alignment module FAM, the model edge detection accuracy is improved, the attention feature enhancement module FEM combined with channel enhancement and spatial enhancement is designed to realize the change information enhancement and remote transmission of the fused feature, the model change feature extraction performance is effectively improved, and high-quality change features are output, and the application of GELU activation function further improves the nonlinear fitting ability of the model. BRIEF DESCRIPTION OF DRAWINGS

[0035] Figure 1 is the overall network structure diagram of the present application;

[0036] Figure 2 is the structure diagram of the semantic flow alignment module of the present application;

[0037] Figure 3 is the structure diagram of the attention feature enhancement module of the present application;

[0038] Figure 4 is the structure diagram of the transpose convolution group combined with the inverted residual of the present application;

[0039] Figure 5 is the training set data example diagram of the present application;

[0040] Figure 6 is the CDD data set detection result diagram of the present application;

[0041] Figure 7is a CDGZ dataset detection example of the present application. DETAILED DESCRIPTION

[0042] The present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0043] As Figure 1 shown, a change detection method based on semantic alignment and feature enhancement proposes a change detection twin network model based on semantic flow alignment and attention feature enhancement module (SACD) for remote sensing image building change detection. The SACD model is constructed with a three-branch feature extractor in the encoding stage with the ResNet34 network structure.

[0044] Specifically, the three-branch structure has the same network structure, in which the two time phase feature extraction branches F1 and F2 share parameters, the fusion feature branch F3 / F is independent, and the fusion feature branch is initially obtained by a double-branch feature value superposition method, then the three-branch features are fused by a semantic flow alignment module (FAM), the low-dimensional pooled features from the same branch are marked as F3, and the fusion results of the other two branches are marked as F. The fusion feature F is enhanced by an attention feature enhancement module (FEM) as remote information and is superimposed with the same dimension information in the decoding stage. In the decoding stage, the resolution is restored to the original image by a transposed convolution group (TCG) combined with an inverted residual layer by layer, and finally the pixel-by-pixel change recognition result is output. Figure 1

[0045] As Figure 2 shown, the module inputs F1, F2, and F3 are time phase 1 feature maps, time phase 2 feature maps, and last level fusion feature down-sampling feature maps, respectively. The double-time phase features F1 and F2 are scheduled with F3 as a guide, and the scales are CxHxW. The three are fused into a feature F.

[0046] ​In-depth, first two branch extraction features F1, F2 are channel spliced and channel invariant convolution operation is performed to realize preliminary information fusion, then F3 is channel spliced and convolution to obtain a position offset map, the position offset map value is the index corresponding to the preliminary information fusion result and the fusion branch position, according to the index information, the semantic alignment of the feature map is realized, the scheduling result is value superimposed with F3 feature after two convolution operations, three branch information fusion is realized, and the fusion process is as follows:

[0047] F = Warp[F cov,Gelu [F cat,conv (F1, F2), F cov (F3)], F cat，conv (F1, F2)] + F cov (F3).

[0048] The SACD model activation function selection combines dropout, region output and input dependence, and is superior to the new type of activation function of ReLU (Rectified Linear Units): Gaussian error linear unit GELU (Gaussian Error Linear Units) in performance.

[0049] The GELU function weights the input by the value of the input, and the function expression is Where is the standard Gaussian cumulative distribution function, which can also be expressed by the Gaussian error function erf(x):

[0050]

[0051] The error function image is relatively close to the function tanh, and a local fitting method is used to construct an approximate expression tanh(ax+bx 3 ) Taylor expansion at x=0 and solve, and the fitting GELU function is defined as:

[0052]

[0053] As Figure 3As shown, the core task of the attention mechanism is to generate weight values of the features from the input features, and the DANet selectively aggregates the features of each position by weighting the features of all positions, and selectively emphasizes the interdependence between channels by integrating the channel mapping of all dimensions. The application constructs an attention feature enhancement module containing channel enhancement and spatial enhancement to obtain better feature representation on this basis, which is different from the original module attention generation which directly uses the original image convolution and multiplication method. The advantages of the proposed module are that in the channel enhancement branch, the spatial information is first aggregated, and in the spatial enhancement branch, the channel information is first integrated, and more effective weight calculation is realized on the basis of feature fusion; two different types of attention mechanisms realize different levels of reinforcement learning on the fused features, and the sum EF of the channel enhanced feature CF, the spatial enhanced feature SF and the input feature F is output, and the superposition of the input feature makes the module tend to learn the feature residual.

[0054] Further, the attention feature enhancement module FEM realizes feature enhancement in the channel dimension through channel attention. First, two groups of feature maps are obtained through 1x1 convolution and 3x3 convolution respectively, wherein V1 completely compresses the channels, and V2 maintains the original resolution. Then, V1 changes the shape to HW1x1x1, and 1x1 convolution and an activation function are used again to obtain the weight Cx1x1, and V2 changes the matrix dimension and shape to HWxC, and 1x1 convolution is used to reduce the dimension to 1xC to aggregate the spatial information, which is equivalent to global pooling. The weight is filtered through an activation function again; the weights obtained in two different ways are multiplied to obtain the channel feature weight CW(CxC), and finally multiplied with the input feature to obtain the channel enhanced feature CF. The channel enhanced feature operation formula is:

[0055] CF=F cov (F)×F Gelu [F cov,Gelu (F reshape (V1))×F cov，Gelu (F reshape (V2))]

[0056] Further, the attention feature enhancement module FEM realizes feature enhancement in the spatial dimension through spatial attention. V3 only changes the shape of the feature matrix to CxHW, and then 1x1 convolution is used to integrate the channel mapping, and an activation function is used for enhancement. V4 first uses 1x1 convolution to realize neighborhood information interaction and channel compression to obtain multi-channel integrated features with a shape of 1xHxW. Shape remodeling (HWx1x1) and activation enhancement are performed on it, and matrix multiplication is performed on the two groups of weights to obtain the spatial feature weight SW(HWxHW) through an activation function again. Multiply the input feature and output the spatial enhanced feature SF through shape remodeling again. The spatial enhanced feature operation formula is:

[0057] SF = F cov (F) x F Gelu [F cov,Gelu (V3) x F Gelu (F reshape (V4))

[0058] In addition, the decoding process is implemented by combining the deconvolution group of the inverted residual module, and the convolution group operation includes three parts of feature accumulation, inverted residual and transpose convolution. The feature accumulation sums the values of the features EF output by the feature enhancement module and the features DF from the high-dimensional deconvolution in the same dimension, realizes the long-distance information transmission of the encoding process and the decoding process; the inverted residual is used to enhance the learning ability of the model, and the transpose convolution realizes the upsampling of the feature map to the low-dimensional feature LDF.

[0059] The commonly used residual structure adopts the "compression-convolution-dilation-stacking" process, first compresses the channel number of the input feature through 1x1 convolution, performs regular 3x3 convolution feature extraction under the compressed channel, and then uses 1x1 convolution to increase the channel number to be consistent with the input feature F, and sums the input feature and the feature value output by the module. The inverted residual adopted by the present application adopts the opposite "dilation-convolution-compression-stacking" process, and the channel number is increased to 3 times the input channel number in the 1x1 convolution, and then reduced and stacked after feature extraction. The inverted residual module can expand the feature information on the one hand in the scene where the input feature channel number is small, optimize the residual calculation ability of the network structure, and on the other hand can promote the model to learn the residual between the feature value and the true value, and avoid the model gradient disappearance.

[0060] The embodiment adopts change detection public evaluation data sets: Season-varying Change Detection Dataset (CDD) and CDGZ. The CDD dataset contains three types of data including object-free relative displacement synthetic images, object small displacement synthetic images, and real remote sensing images of seasonal changes obtained by Google Earth (Digital Globe), a total of 11 pairs, all of which are three-band (R, G, B) images, of which 7 pairs of images contain real seasonal changes. The spatial resolution of these images is 3cm / px to 100cm / px, a total of 16,000 pairs of images, with a size of 256x256 pixels, and after allocation, the training set has 10,000 image pairs, and the verification set and the test set each have 3,000 image pairs.

[0061] The training set example is shown in Figure 5 The marked change types mainly include cars, buildings, roads, fences, etc., and the number of change instance objects pixel groups varies from dozens to tens of thousands. The main interference factors include imaging errors, seasonal vegetation changes, and ice and snow coverings.

[0062] CDGZ is a high-resolution satellite image change detection dataset collected by BIGEMAP software using Google Earth services, covering the suburbs of Guangzhou City, China, from 2006 to 2019, with a spatial resolution of 0.55m, a size range of 1006x1168 to 4936x5224 pixels, and a total of 20 pairs of VHR image pairs for seasonal changes. 10000 pairs of 256x256 pixel images were generated by first non-overlapping cropping and then random cropping, with 7500 image pairs in the training set, 1500 and 1000 image pairs in the validation and test sets respectively, and only building changes were marked in this dataset.

[0063] The networks trained in this application are all implemented on the Python-based open source deep learning framework Pytorch 1.7.0, with a device of RTX 2080Ti. All network hyperparameters are uniformly set, with an initial learning rate of 0.01, a network parameter update optimizer of Adam, a label and model output difference evaluation loss function of BCEloss, and a Batchsize of 4.

[0064] The experimental comparison network in this application selects FC-Siam-conc, FC-Siam-diff, CosimNet, BidateNet, IFN, and DASNet, which uses DANet attention module, similar to the network structure in this paper. The change detection results select four evaluation indexes of F1 value, precision, recall rate and overall accuracy, where TP pixels are true predictions, TN pixels are false predictions, FP pixels are false predictions, and FN pixels are false predictions. The evaluation indexes are as follows:

[0065]

[0066] The experimental results are shown in Tables 1 and 2. The best performing comparison models on the two datasets are DASNet and CosimNet, with F1 values of 92.39% and 94.06% respectively. The proposed model SACD has the best performance on both CDD and CDGZ datasets, with F1 values 2.5% and 2.34% higher than the second highest value model respectively. The model design flow alignment module FAM realizes three-branch information fusion, optimizes semantic accuracy, and uses the attention feature enhancement module FEM to realize change feature fusion, enhancement and transmission integration, and strengthen change characteristic learning, effectively improving model precision.

[0067] Table 1 Accuracy comparison of CDD dataset (%)

[0068] Model F1 value Accuracy Recall Total accuracy FC-Siam-conc 58.93 95.96 42.52 92.68 FC-Siam-diff 62.13 94.04 46.39 93.02 CosimNet 87.55 91.25 84.14 97.05 BidateNet 90.67 97.84 84.48 97.85 IFN 91.26 98.72 84.85 97.99 DASNet 92.39 94.39 90.47 98.16 SACD 94.71 98.87 90.89 98.75

[0069] Table 2 Accuracy comparison of CDGZ dataset (%)

[0070] Model F1 value Accuracy Recall Total accuracy FC-Siam-conc 71.27 92.90 57.80 95.76 FC-Siam-diff 69.13 93.44 54.86 95.54 CosimNet 94.06 93.02 95.12 98.74 BidateNet 87.81 94.64 81.90 97.93 IFN 80.93 94.93 70.53 96.97 DASNet 90.41 95.55 85.79 98.34 SACD 96.40 96.85 95.96 99.35

[0071] Figure 6 、 7 Qualitative analysis of each model change detection result is performed by selecting typical regions of CDD and CDGZ datasets respectively, and the performance difference parts are marked with red boxes in each region label.

[0072] Figure 6 is the detection result of CDD dataset, mainly comparing the detection performance difference of models in small area details. Region 1 has large light difference, shadow blocking, and obvious vegetation change due to ice and snow coverage, and the main change markers are vehicles and roads. The comparison models FC-Siam-conc and FC-Siam-diff do not learn the change characteristics at all, the other four methods can effectively detect road changes but cannot effectively output the complete edge of the vehicle, and SACD can identify and output the vehicle in a relatively regular and complete manner; Regions 2 and 3 are also affected by ice and snow coverage, and the main change targets are residential building groups and newly built roads. Each comparison model has significant over-extraction at the building connection in region 2, and in region 3, the road color is similar to the background color, and DASNet performs well, but the road is discontinuous and incomplete, and the other models have a large number of missing extractions, and SACD has the least missing and false extractions; Region 4 mainly compares the multi-scale change target detection ability of the models, and there is a large area of change in the region, and each model can extract well, but it performs poorly in distinguishing between the long fence and the building.

[0073] Figure 7 is a detection example of CDGZ dataset, and a large-scale scale is selected to analyze the comprehensive detection ability of the model in complex scenes. The buildings in the red boxes in regions 1 and 3 are not completely extracted by the comparison models, there are a small number of holes in the buildings in region 2, and there is significant missing extraction in region 4, and the extraction result of SACD is closest to the label. In summary, the extraction result of the proposed model SACD is the best in terms of completeness, and it performs best among all models. The addition of the attention feature enhancement module effectively improves the extraction, enhancement and fusion ability of the model for change information, realizes high-quality change recognition, and the flow alignment module gives the model strong semantic learning and edge extraction ability.

[0074] In summary, the principle of the embodiment is that in the encoding process, the change characteristics are extracted by using a three-branch structure, the edge alignment is performed between different branches through the semantic flow alignment module FAM, then the attention feature enhancement module FEM is designed in the skip connection to perform feature fusion and enhancement from the channel and space respectively, the output result is superimposed with the deep feature as the remote information, and the residual connection is used to output the pixel-by-pixel change detection result through the transconvolution group TCG.

[0075] The specific embodiments described herein are merely illustrative of the principles of this application. Numerous modifications or adaptations will be readily apparent to those skilled in the art of this application without departing from the spirit or scope of the application as defined by the following claims.

[0076] Although the terms FAM, EFM, TCG, etc. are used herein more frequently, the possibility of using other terms is not excluded. The use of these terms is merely for the convenience of describing and explaining the nature of the application; any interpretation of them as an additional limitation is contrary to the spirit of the application.

Claims

1. A method for change detection based on semantic alignment and feature enhancement, characterized in that, Comprise the following steps: S1: in the encoding stage, a three-branch feature extractor is constructed with the ResNet34 network structure, wherein the parameters of two time-phase feature extraction branches are shared, and the parameters of the fusion feature branch are independent; S2: the three-branch features are fused and scheduled by a semantic flow alignment module FAM, the low-dimensional pooled features from the same branch are marked as F3, and the fusion results of the other two branches are marked as F; S21: the two branch extraction features F1 and F2 are channel spliced and channel invariant convolution operation is performed to realize preliminary apparent fusion, and then channel splicing and convolution are performed with F3 to obtain a position offset map; S22: the offset map value is the index corresponding to the preliminary information fusion result and the fusion branch position, and the index information is scheduled to realize semantic alignment of the feature map, the scheduling result is superimposed with the F3 feature after two convolution operations, and three-branch information fusion is realized, and the fusion process is as follows: F = Warp[F cov,Gelu [F cat,conv (F1,F2),F cov (F3)],F cat,conv (F1,F2)]+F cov (F3); S3: the fused feature F is enhanced by an attention feature enhancement module FEM and superimposed with the same dimension information in the decoding stage; the attention feature enhancement module FEM in step S3 first aggregates spatial information in the channel strengthening branch, and then integrates channel information in the spatial strengthening branch, and after feature fusion, weight calculation is performed, and channel enhancement features CF and spatial enhancement features S are output, and the sum EF of the three is obtained by superimposing the input feature F; S4: in the decoding stage, the resolution is restored to the same as the original image layer by layer by combining the transpose convolution group TCG with the inverted residual, and finally the pixel-by-pixel change recognition result is output.

2. The method of claim 1, wherein, The initial fusion feature branch in step S1 is obtained by double-branch feature value superposition.

3. The method of claim 1, wherein, The activation function of step S2 adopts Gaussian error linear unit GELU.

4. The method of claim 1, wherein, The module inputs F1, F2 and F3 in step S2 are time phase 1 feature map, time phase 2 feature map and down-sampled feature map of the last level fusion feature, respectively, and the double-time-phase features F1 and F2 are scheduled with F3 as the guide, and the scales are all CxHxW, and the three are fused into a feature F.

5. The method of claim 1, wherein, The attention feature enhancement module FEM in step 3 realizes feature enhancement in channel dimension through channel attention, and the specific steps are as follows: S31: two groups of feature maps V1 and V2 are obtained through 1x1 convolution and 3x3 convolution, wherein V1 completely compresses the channel, and V2 maintains the original resolution; S32: V1 changes the shape to HW1x1x1, and then 1x1 convolution and activation function are used to obtain weight Cx1x1, and V2 changes the matrix dimension and shape to HWxC, and then 1x1 convolution is used to reduce the dimension to 1xC to aggregate spatial information, and HW position information is included in the same pixel, and then an activation function is used to filter the weight; S33: the weights obtained in two different ways perform matrix multiplication, and then an activation function is used to obtain channel feature weight CW(CxC), and finally the channel enhancement feature CF is obtained by multiplying the input feature, and the channel enhancement feature operation formula is: CF = F cov (F) x F Gelu [F cov,Gelu (F reshape (V1)) x F cov,Gelu (F reshape (V2))].

6. The method of claim 5, wherein, The attention feature enhancement module FEM in step 3 realizes feature enhancement in spatial dimension through spatial attention, and the specific steps are as follows: S34: V3 changes the feature matrix shape to CxHW, and then maps and integrates the channels through 1x1 convolution, and enhances it with an activation function, V4 first adopts 1x1 convolution to realize neighborhood information interaction and channel compression, obtains multi-channel integrated features, and the shape is 1xHxW, and then reshapes (HWx1x1) and enhances it with an activation function; S35: The two groups of weights perform matrix multiplication and obtain spatial feature weights SW (HWxHW) again through an activation function, multiply them with the input features, reshape them again, and output spatial enhancement features SF, and the spatial enhancement feature operation formula is: SF = F cov (F) x F Gelu [F cov,Gelu (V3) x F Gelu (F reshape (V4))].

7. The method of claim 6, wherein, The transpose convolution group TCG combined with the inverted residual in step 4 comprises the following steps: S41: Adopting feature accumulation, summing the values of the high-dimensional deconvolution features DF and the feature enhancement module output features EF in the same dimension to realize long-distance information transmission of the encoding process and the decoding process; S42: Using the learning ability of the inverted residual enhancement model; S43: Using transpose convolution to realize up-sampling of the feature map to low-dimensional features LDF.

8. The method of claim 7, wherein, In step S42, the inverted residual first increases the number of channels in the 1x1 convolution to 3 times the input channel number, and then reduces the dimension after feature extraction.

Images