Double-branch change detection method and system based on visual basic model

Abstract

The invention discloses a double-branch change detection method and system based on a visual basic model, and belongs to the technical field of remote sensing image detection. According to the method, a double-branch change detection network model based on a visual basic model is provided, and the frozen visual basic model FastSAM is used as an encoder, so that the feature extraction capability of the network model is greatly improved. In order to realize migration of extracted features of a visual basic model to the field of remote sensing images, a change detection adapter module is designed in the method, and effective conversion from universal visual features to remote sensing image features is realized through fine tuning of a small number of model parameters. In addition, the method also provides a double-branch change detection module, a single-temporal semantic feature decoder and a double-temporal change feature decoder are respectively utilized, and through mining of single-temporal remote sensing image semantic information and fusion of double-temporal multi-scale features, the feature discrimination capability is further improved, so that the detection performance of the model is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of remote sensing image detection, and in particular relates to a dual-branch change detection method and system based on a visual basic model. Background Art

[0002] Remote sensing image change detection is an important technology used in the interpretation and analysis of high-resolution remote sensing image data. This technology aims to use two or more remote sensing images and related geospatial data covering the same surface area at different times, combined with the corresponding ground object characteristics and remote sensing imaging mechanism, and use image processing algorithms and mathematical model methods to determine and analyze the ground object changes in the region within a specific period. It has important value and significance for practical applications such as natural resource management and national land space planning. Since the 1980s, remote sensing image change detection methods have experienced rapid development. In particular, thanks to the rapid progress of deep learning technology, remote sensing image change detection technology has achieved significant improvements in detection accuracy and has been widely used. However, remote sensing image change detection methods based on deep learning still face some challenges. Specifically, the existing deep learning models still need to be improved in feature extraction capabilities. In addition to having accurate spatial positioning capabilities, they also need to understand the semantic information corresponding to pixels. The limitations of existing methods in feature extraction capabilities lead to their unsatisfactory performance in processing complex scenes and understanding different types of change areas, which significantly limits their application in practical scenarios. Therefore, it is urgent to improve the feature extraction capabilities of deep learning models.

[0003] Recently, visual foundation models represented by SAM have demonstrated powerful feature extraction capabilities and zero-shot generalization capabilities, and are considered a new milestone in the development of computer vision technology. By pre-training on massive natural scene image data, SAM can recognize and segment arbitrary objects through prompt signals such as points, boxes, and texts, showing a broad understanding of various objects and scenes. With the introduction of SAM, more visual foundation models, such as EfficientSAM, FastSAM, and MobileSAM, have also been introduced. These models are not only comparable to SAM in performance, but also greatly improve the inference efficiency. Inspired by these methods, the powerful feature extraction capabilities of advanced visual foundation models are used to improve existing change detection methods, showing great potential. However, although visual foundation models perform well in a variety of natural scenes, there are still significant problems in directly migrating them to downstream tasks in specific fields, especially in highly specialized fields such as medical imaging and remote sensing images. Specifically, the visual base model is usually pre-trained on natural images on a large scale, but there are significant differences in image features and distribution between natural images and remote sensing images, and the visual base model lacks a deep understanding of remote sensing image features and remote sensing knowledge. Therefore, it is often difficult to achieve ideal results when directly applied to remote sensing image change detection tasks.

[0004] In summary, although the change detection method based on deep learning has made progress to a certain extent, there is still the problem of insufficient feature extraction capability; and although the visual basic model has powerful feature extraction capabilities, it also faces many challenges when directly applied to remote sensing image change detection due to domain differences. Summary of the invention

[0005] In order to solve these problems, the present invention proposes a dual-branch change detection method based on a visual base model. This method uses the visual base model as an encoder to give full play to its advantages in feature extraction. Then, by designing a change detection adapter module, the migration from general visual features to remote sensing image features is realized, and a dual-branch change detection module is introduced to further improve the change detection performance. This design can not only alleviate the problem of insufficient feature extraction of existing deep learning methods in remote sensing image change detection, but also effectively migrate the features extracted by the visual base model to the remote sensing image field, thereby improving the recognition accuracy of the changed area.

[0006] In view of this, the purpose of the present invention is to provide a dual-branch change detection method based on a visual base model. The method proposes a dual-branch change detection network model based on a visual base model, uses a frozen visual base model FastSAM as an encoder, and uses its excellent feature extraction ability to extract the common features of dual-phase remote sensing images respectively, which greatly improves the feature extraction ability of the network model. In order to realize the migration of the features extracted by the visual base model to the field of remote sensing images, the method designs a change detection adapter module, and realizes the effective conversion from general visual features to remote sensing image features by fine-tuning a small number of model parameters. In addition, the method also proposes a dual-branch change detection module, which respectively uses a single-phase semantic feature decoder and a dual-phase change feature decoder, and further improves the feature discrimination ability by mining the semantic information of single-phase remote sensing images and fusing dual-phase multi-scale features, thereby improving the detection performance of the model. The invention effectively solves the shortcomings of remote sensing image change detection algorithms based on deep learning in feature extraction, and at the same time realizes the effective migration of the visual base model to the field of remote sensing images, further improving the accuracy and robustness of change detection.

[0007] To achieve the purpose of the above invention, the technical solution adopted by the present invention is: a remote sensing image change detection method based on a visual basic model, comprising the following steps:

[0008] Step 1), collect and process the remote sensing image change detection dataset required for training the model;

[0009] Step 2), using a dual-branch change detection network based on a visual base model to train a remote sensing image change detection model for remote sensing image change detection on the data set in step 1);

[0010] The processing process of the remote sensing image change detection model is as follows: the visual basic model is used as an encoder to extract the universal visual features of the dual-temporal image as the input of the change detection adapter module; the change detection adapter module adjusts the input features to achieve the migration of universal visual features to remote sensing image features, and uses the adjusted features as the input of the dual-branch change detection module; the dual-branch change detection module uses the single-temporal semantic feature decoder branch and the dual-temporal change feature decoder branch for the input features, outputs the change detection results, and supervises the results; finally, the results output by the dual-branch change detection module are weighted fused to output the final detection results;

[0011] Step 3), based on the final remote sensing image change detection model, the changed area of ​​the dual-temporal remote sensing image to be detected is identified and visualized.

[0012] Further, the specific implementation of step 1) is as follows:

[0013] Firstly, the dual-temporal remote sensing image data required for training the remote sensing image change detection model are collected, and the dual-temporal images are calibrated and preprocessed; secondly, the collected data are manually labeled according to the predetermined land cover change types to obtain the labeled files containing the changed areas and change types in the dual-temporal remote sensing images.

[0014] Furthermore, the visual basic model is a FastSAM model.

[0015] Furthermore, the specific processing process of the change detection adapter module is:

[0016] The current feature map is used as input. The number of channels of the input feature is first reduced to one-fourth of the original through 1×1 convolution, while keeping the spatial dimension unchanged. The adjusted features are then input to the normalization layer. Finally, the ReLU activation function is used to perform nonlinear transformation on the normalized features, and the final adjusted features that are more suitable for remote sensing image change detection are output. The feature adjustment method is shown in formula (1):

[0017]

[0018] Specifically, P i Represents the features extracted by the FastSAM encoder, represents the change detection adapter module, F i It represents the features after the adapter module is adjusted, Conv is a 1×1 convolution layer, and BN is a batch normalization function.

[0019] Furthermore, the single-phase semantic feature decoder branch includes a single-phase semantic feature decoder module and a change detection module, and the specific structure is as follows:

[0020] 33) The single-phase semantic feature decoder module includes several decoding units, each of which consists of an upsampling layer, two convolution layers with a step size of 1 and a convolution kernel size of 3×3, two batch normalization layers BN, and two ReLU activation functions. Each decoding unit receives input from high-level features, first performs a deconvolution operation on the current input features, and then concatenates the upsampled features with the feature maps from the lower level to achieve multi-scale feature fusion; through layer-by-layer upsampling and feature fusion, the semantic feature D1 of the previous phase fusion of the multi-scale information is output. The process is shown in formula (2):

[0021] D i =F i , D i-1 =Conv[F i-1 ,Upsample(D i )] (2)

[0022] Specifically, D i represents the i-th layer feature of the single-phase semantic feature decoder, F i Represents the features of different scales after the adapter module is adjusted, Upsample represents the upsampling operation, and Conv represents the convolution layer with a convolution kernel size of 3×3;

[0023] Similarly, the semantic feature D′1 of the posterior phase fusion of the multi-scale information is obtained;

[0024] 34) Next, a change detection module consisting of two layers of convolution, two layers of batch normalization, two layers of ReLU activation function and a Sigmoid function is used to splice the image features of the front and back single-phase images as input, and the change detection results are predicted to obtain the change detection results of the single-phase semantic feature decoder branch. In summary, the calculation process of this branch can be expressed by formula (3):

[0025]

[0026] Specifically, The subscripts 1 and 2 in the table refer to the phases, 1 represents the anterior phase and 2 represents the posterior phase. It represents the feature map of the remote sensing image before and after the output of the change detection adapter after adjustment, M θ (·) represents the single-temporal semantic feature decoder branch.

[0027] Furthermore, the cross entropy loss is used to predict the results For supervision, the loss function is calculated as shown in formula (4):

[0028]

[0029] Specifically, N is the number of samples, C is the number of categories, and y i , c represents the indicator function of the true label of the i-th sample under category c, if it is this category, it is 1, otherwise it is 0, and p i .c is the predicted probability of the i-th sample under category c.

[0030] Furthermore, the specific structure of the dual-phase change feature decoder is as follows:

[0031] 52) The dual-phase change feature decoder includes a branch containing several decoding units, each of which consists of two convolutional layers with a convolution kernel size of 3×3, two batch normalization layers BN and two layers of ReLU activation functions; in the process of decoding the dual-phase change feature, each decoding unit concatenates and fuses the extracted features of the same scale and the upsampled features of a low-scale feature, where the features of the same scale come from the outputs of the previous and next phase features and other decoding units, and the low-scale features come from the outputs of the later phase features and other decoding units; the fused feature map is further extracted through the convolution block to improve the feature expression ability of the network; after the decoding operation of all decoding units, 4 features R containing shallow and deep information are finally output. 1，1 , R 1，2 , R 1，3 and R 1，4 , the decoding process is shown in formula (5):

[0032]

[0033] Specifically, R i，j represents the output of the corresponding position decoding unit, F i Indicates the features from the previous phase, F′ i Represents features from the later phase, Upsample represents the upsampling operation, and Conv represents the convolution layer with a convolution kernel size of 3×3;

[0034] 52) Next, the obtained features are input into a change detection module consisting of a channel attention module CAM, a convolutional layer and a sigmoid function; wherein CAM obtains the global information of each channel through global average pooling and global maximum pooling, then generates channel weights using a fully connected layer, and finally applies these weights to the original feature map, thereby achieving adaptive adjustment of the features; the change detection module receives the concatenated features of the four features output by the dual-phase change feature decoder in the channel dimension as input, and outputs the change detection result of the dual-phase change feature decoder branch In summary, the calculation process of this branch can be expressed by formula (6) as follows:

[0035]

[0036] Specifically, The subscripts 1 and 2 in the table refer to the phases, 1 represents the anterior phase and 2 represents the posterior phase. The feature map representing the before and after phase remote sensing images output by the change detection adapter after adjustment. Represents the dual-phase-change feature decoder branch.

[0037] Furthermore, the calculation formula for weighted fusion of the change detection results output by the dual-branch change detection module is as follows:

[0038]

[0039] Specifically, is the change detection result of the single-phase semantic feature decoder branch, is the change detection result of the dual-phase change feature decoder branch, Conv 1×1 (·) represents 1×1 convolution operation, and Cat(·) represents channel concatenation.

[0040] Furthermore, during the training process, cross entropy loss is used to predict the results. or To supervise.

[0041] The present invention also provides a dual-branch change detection system based on a visual basic model, including a processor and a memory, the memory is used to store program instructions, and the processor is used to call the stored instructions in the memory to execute a dual-branch change detection method based on a visual basic model as described in the above technical solution.

[0042] Compared with the prior art, the advantages and beneficial effects of the present invention are as follows:

[0043] 1) The present invention uses the visual basic model FastSAM to extract the feature information of the image. Compared with the existing technology, it has stronger feature extraction ability and effectively improves the recognition accuracy of the changed area.

[0044] 2) The present invention successfully migrates the universal visual features extracted by the visual basic model FastSAM to the field of remote sensing images through the change detection adapter module. In addition, the dual-branch change detection module in the present invention can respectively utilize a single-phase semantic feature decoder and a dual-phase change feature decoder, and further improves the feature discrimination ability by mining the semantic information of single-phase remote sensing images and fusing dual-phase multi-scale features, thereby improving the detection performance of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 Schematic diagram of a flow chart for implementing the present invention.

[0046] Figure 2 A diagram of the branches of a single-phase semantic feature decoder implemented in the present invention.

[0047] Figure 3 It is a diagram of the branches of the dual phase change feature decoder implemented in the present invention.

[0048] Figure 4The change detection effects of the present invention and other change detection methods on the WHU-CD dataset are compared. DETAILED DESCRIPTION

[0049] In order to facilitate ordinary technicians in the field to understand and implement the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the implementation examples described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0050] like Figure 1 The technical solution adopted by the present invention is a remote sensing image change detection method based on a visual basic model, comprising the following steps:

[0051] (1) In the encoder module, the visual base model FastSAM is used to perform universal feature extraction on the input dual-phase remote sensing image to obtain the multi-scale feature map of the before and after phases, which is used as the input of the change detection adapter module;

[0052] (2) The change detection adapter module fine-tunes the general visual features extracted by the visual basic model FastSAM, thereby achieving effective conversion of general visual features to remote sensing image features, and uses the converted features as the input of the dual-branch change detection module;

[0053] (3) The dual-branch change detection module uses a single-phase semantic feature decoder and a dual-phase change feature decoder respectively. By mining the semantic information of the single-phase remote sensing image and fusing the dual-phase multi-scale features, the detection results of the two branches are obtained. Finally, the results of the two branches are fused to output the final change detection result.

[0054] The specific implementation steps are as follows:

[0055] 1) Input a dual-temporal remote sensing image and use the visual basic model FastSAM to extract convolution feature maps of different spatial resolutions. In this embodiment, convolution feature maps at four scales are obtained for each phase image;

[0056] 2) Using the change detection adapter module to adjust each convolution feature map, we can obtain the convolution feature maps of the previous and next phases that are more suitable for the remote sensing image change detection field, and use them as the input features of the dual-branch change detection module;

[0057] The change detection adapter module takes the feature map of each scale as input, adjusts the number of channels of the input feature map to one-fourth of the original through a layer of convolution with a kernel size of 1×1, and then inputs the adjusted features into the normalization layer to accelerate model training and alleviate overfitting. Finally, the ReLU activation function is used to perform nonlinear transformation on the normalized features, so that the model can better fit complex functional relationships and output the final adjusted features that are more suitable for the field of remote sensing image change detection.

[0058] The specific processing process of the change detection adapter module in step 2) is to use the current feature map as input, first reduce the number of channels of the input feature through 1×1 convolution, and adjust it to one-fourth of the original, while keeping the spatial dimension unchanged, and then input the adjusted features into the normalization layer to accelerate model training and alleviate overfitting. Finally, the ReLU activation function is used to perform nonlinear transformation on the normalized features, so that the model can better fit the complex functional relationship, and output the final adjusted features that are more suitable for the field of remote sensing image change detection. The feature adjustment method is shown in formula (1):

[0059]

[0060] Specifically, P i Represents the features extracted by the FastSAM encoder, represents the change detection adapter module, F i It represents the features after the adapter module is adjusted, Conv is a 1×1 convolution layer, and BN is a batch normalization function.

[0061] 3) The convolution feature maps of the previous and next phases adjusted by the change detection adapter in 2) are used as the input of the single-phase semantic feature decoder branch, and a single-phase semantic feature decoder unit is established by using a stepwise upsampling and fusion method. The unit decodes the features of the previous and next phases separately to obtain the semantic feature maps of the previous and next phases, and uses them as the input of the subsequent change detection unit;

[0062] like Figure 2 As shown in Figure 2, the specific details of the single-phase semantic feature decoder branch are as follows:

[0063] 1) This branch first includes a single-phase semantic feature decoder module, which contains three decoding units. Each decoding unit consists of an upsampling layer, two convolution layers with a step size of 1 and a convolution kernel size of 3×3, two batch normalization layers (BN), and two ReLU activation functions. Each decoding unit receives input from high-level features, first performs a deconvolution operation on the current input features, and then concatenates the upsampled features with the feature maps from the lower level to achieve multi-scale feature fusion. Then, the above steps are repeated to gradually restore spatial information and improve feature expression capabilities. The decoding method is shown in formula (2):

[0064] D4=F4,D i-1 =Conv[F i- 1. Upsample(D i )] (2)

[0065] Specifically, D i represents the i-th layer feature of the decoder, F i Indicates features of different scales, F4 is a feature map at a scale of 1 / 32 of the original image size, Upsample represents an upsampling operation, and Conv represents a convolution layer with a convolution kernel size of 3 × 3. Through layer-by-layer upsampling and feature fusion, the semantic feature D1 of the fusion of each multi-scale information is output.

[0066] Similarly, according to the above method, the semantic feature D′1 of the posterior phase fusion of the respective multi-scale information is obtained;

[0067] 2) After the single-phase semantic feature decoder module, there is a change detection module consisting of two layers of convolution, two layers of batch normalization, two layers of ReLU activation function and a Sigmoid function. The image features of the front and rear single-phase images are fused and spliced ​​as input, and the change detection results are predicted to obtain the change detection results of the single-phase semantic feature decoder branch. The calculation process of this branch can be expressed by formula (3):

[0068]

[0069] Specifically, The subscripts 1 and 2 in the table refer to the phases, 1 represents the anterior phase and 2 represents the posterior phase. It represents the feature map of the remote sensing image before and after the output of the change detection adapter after adjustment, M θ (·) represents the single-phase semantic feature decoder branch. The cross entropy loss is used to predict the results during training. For supervision, the loss function is calculated as shown in formula (4).

[0070]

[0071] Specifically, N is the number of samples, C is the number of categories, and y i , c represents the indicator function of the true label of the i-th sample under category c (if it is this category, it is 1, otherwise it is 0), and p i .c is the predicted probability of the i-th sample under category c predicted by the model.

[0072] 4) Perform change detection on the semantic feature graphs of the previous and next phases, and supervise the change detection results to enhance the model's deep semantic understanding of the changed area, thereby improving the model's ability to capture change details and its robustness in different scenarios;

[0073] 5) The convolution feature maps of the before and after phases adjusted by the change detection adapter in 2) are used as the input of the dual-phase change feature decoder branch, and the change information of all scales of the before and after phases is integrated by layer-by-layer splicing and fusion to obtain the change feature maps of all scales of the fused before and after phases, and used as the input of the subsequent change detection unit;

[0074] like Figure 3 As shown, the specific details of the dual-phase change feature decoder branch are as follows:

[0075] 1) This branch contains a dual-phase change feature decoder module, which contains 10 decoding units, each of which consists of two convolutional layers with a convolution kernel size of 3×3, two batch normalization layers (BN) and two layers of ReLU activation functions. In the process of decoding the dual-phase change features, each decoding unit concatenates and fuses the extracted features of the same scale and the upsampled features of a low-scale feature, where the features of the same scale come from the outputs of the previous and next phase features and other decoding units, and the low-scale features come from the outputs of the later phase features and other decoding units. The fused feature map is further extracted by the convolution block to improve the feature expression ability of the network. After the decoding operation of all decoding units, 4 features R containing shallow and deep information are finally output. 1，1 , R 1，2 , R 1，3 and R 1，4 , the decoding process is shown in formula (5):

[0076]

[0077] Specifically, R i，j represents the output of the corresponding position decoding unit, F i Indicates the features from the previous phase, F′ iRepresents the features from the later phase, Upsample represents the upsampling operation, and Conv represents the convolution layer with a convolution kernel size of 3×3.

[0078] 2) After the dual-phase change feature decoder, there is a change detection module consisting of a channel attention module (ChannelAttentionModule, CAM), a convolutional layer and a Sigmoid function. Among them, CAM obtains the global information of each channel through global average pooling and global maximum pooling, and then uses the fully connected layer to generate channel weights, and finally applies these weights to the original feature map to achieve adaptive adjustment of the features. In this way, CAM can effectively improve the model's attention to key features, improve the model's expressiveness and performance. The change detection module receives the concatenated features of the four features output by the dual-phase change feature decoder in the channel dimension as input, and outputs the change detection results of the dual-phase change feature decoder branch. The calculation process of this branch can be expressed by formula (6) as follows:

[0079]

[0080] Specifically, The subscripts 1 and 2 in the table refer to the phases, 1 represents the anterior phase and 2 represents the posterior phase. The feature map representing the before and after phase remote sensing images output by the change detection adapter after adjustment. Represents the branch of the dual-phase feature decoder. The cross entropy loss is used to predict the results during training. For supervision, the loss function is calculated as shown in formula (4).

[0081]

[0082] Specifically, N is the number of samples, C is the number of categories, and y i .c represents the indicator function of the true label of the i-th sample under category c (1 if it is this category, otherwise 0), and p i .c is the predicted probability of the i-th sample under category c predicted by the model.

[0083] 6) Perform change detection on the change feature map that integrates all scale information of the previous and next phases, and supervise the obtained change detection results to improve the model's perception of the relationship between detail changes and global context;

[0084] 7) The change detection results outputted from 4) and 6) are weightedly fused, and the obtained change detection is used as the final output result of the whole method. This result shows excellent performance in the accuracy of identifying the changed area.

[0085] In practical applications, in order to improve the performance and robustness of change detection results, this method introduces a weighted fusion module after the dual-branch change detection module, that is, the detection results of the single-phase semantic feature change detection branch and the dual-phase change feature detection branch are combined. and Perform channel splicing and weighted fusion, and calculate the results It is output as the final change detection result of this method. The calculation formula of the weighted fusion step is as follows:

[0086]

[0087] Specifically, Conv 1×1 (·) indicates 1×1 convolution operation, and Cat(·) indicates channel concatenation. During the training process, cross entropy loss is used to predict the results. For supervision, the loss function is calculated as shown in formula (4).

[0088]

[0089] Specifically, N is the number of samples, C is the number of categories, and y i .c represents the indicator function of the true label of the i-th sample under category c (1 if it is this category, otherwise 0), and p i .c is the predicted probability of the i-th sample under category c predicted by the model.

[0090] Figure 4 To compare the change detection effects of the present invention with other change detection methods on the WHU-CD dataset, the dual-branch change detection model based on the visual basic model proposed in the present invention can make full use of the visual basic model FastSAM to improve the feature extraction ability of the model. In addition, thanks to the dual-branch change detection module, the discriminative and representational capabilities of the features are improved, and the semantic information and change area information contained in the multi-scale features are fully mined, so that the model shows higher detection accuracy. Table 1 is the quantitative comparison results of the method of the present invention and other change detection methods on the WHU-CD dataset. The evaluation indicators used in the table are precision (P), recall (R) and F1 value (F1), and the F1 value is used as the final reference indicator of the model performance.

[0091] Table 1 Quantitative comparison results of the present invention and other change detection methods on the WHU-CD dataset

[0092]

[0093] As can be seen from Table 1, the F1 score of the model trained by the method of the present invention surpasses all other methods, reaching 92.91%, and exceeds the FTN model with the highest F1 score by 0.7%. The experimental results show that the method proposed in the present invention shows a more excellent performance on the WHU-CD dataset.

[0094] On the other hand, an embodiment of the present invention also provides a dual-branch change detection system based on a visual basic model, including a processor and a memory, the memory being used to store program instructions, and the processor being used to call the stored instructions in the memory to execute a dual-branch change detection method based on a visual basic model as described in the above technical solution.

[0095] It should be understood that the above description of the preferred embodiment is relatively detailed and cannot be regarded as limiting the scope of patent protection of the present invention. Under the enlightenment of the present invention, ordinary technicians in this field can also make substitutions or modifications without departing from the scope of protection of the claims of the present invention, which all fall within the scope of protection of the present invention. The scope of protection requested for the present invention shall be based on the attached claims.

Claims

1. A dual-branch change detection method based on a visual basis model, characterized in that: The steps include: Step 1), collect and process the remote sensing image change detection dataset required for training the model; Step 2), using a dual-branch change detection network based on a visual base model to train a remote sensing image change detection model for remote sensing image change detection on the data set in step 1); The processing process of the remote sensing image change detection model is as follows: the visual basic model is used as an encoder to extract the universal visual features of the dual-temporal image as the input of the change detection adapter module; the change detection adapter module adjusts the input features to achieve the migration of universal visual features to remote sensing image features, and uses the adjusted features as the input of the dual-branch change detection module; the dual-branch change detection module uses the single-temporal semantic feature decoder branch and the dual-temporal change feature decoder branch for the input features, outputs the change detection results, and supervises the results; finally, the results output by the dual-branch change detection module are weighted fused to output the final detection results; Step 3), based on the final remote sensing image change detection model, the changed area of ​​the dual-temporal remote sensing image to be detected is identified and visualized.

2. The dual-branch change detection method based on a visual base model as claimed in claim 1, characterized in that: The specific implementation of step 1) is as follows: Firstly, the dual-temporal remote sensing image data required for training the remote sensing image change detection model are collected, and the dual-temporal images are calibrated and preprocessed; secondly, the collected data are manually labeled according to the predetermined land cover change types to obtain the labeled files containing the changed areas and change types in the dual-temporal remote sensing images.

3. The dual-branch change detection method based on a visual base model as claimed in claim 1, characterized in that: The visual basic model is the FastSAM model.

4. The dual-branch change detection method based on a visual base model as claimed in claim 1, characterized in that: The specific processing of the change detection adapter module is: Take the current feature map as input, first reduce the number of channels of the input feature through 1×1 convolution, adjust it to one-fourth of the original, while keeping the spatial dimension unchanged, then input the adjusted features into the normalization layer, and finally use the ReLU activation function to perform nonlinear transformation on the normalized features, and output the final adjusted features that are more suitable for the field of remote sensing image change detection; The feature adjustment method is shown in formula (1): Specifically, P i Represents the features extracted by the FastSAM encoder, represents the change detection adapter module, F i It represents the features after the adapter module is adjusted, Conv is a 1×1 convolution layer, and BN is a batch normalization function.

5. The dual-branch change detection method based on a visual base model as claimed in claim 1, characterized in that: The single-phase semantic feature decoder branch includes a single-phase semantic feature decoder module and a change detection module. The specific structure is as follows: 31) The single-phase semantic feature decoder module includes several decoding units, each of which consists of an upsampling layer, two convolution layers with a step size of 1 and a convolution kernel size of 3×3, two batch normalization layers BN, and two ReLU activation functions. Each decoding unit receives input from high-level features, first performs a deconvolution operation on the current input features, and then concatenates the upsampled features with the feature maps from the lower level to achieve multi-scale feature fusion; through layer-by-layer upsampling and feature fusion, the semantic feature D1 of the previous phase fusion of the multi-scale information is output. The process is shown in formula (2): D i =F i , D i-1 =Conv[F i-1 ,Upsample(D i )] (2) Specifically, D i represents the i-th layer feature of the single-phase semantic feature decoder, F i Represents the features of different scales after the adapter module adjusts them, Upsample represents the upsampling operation, and Conv represents the convolution layer with a convolution kernel size of 3×3; Similarly, the semantic feature D′1 of the posterior phase fusion of the multi-scale information is obtained; 32) Next, a change detection module consisting of two layers of convolution, two layers of batch normalization, two layers of ReLU activation function and a Sigmoid function is used to splice the image features of the front and back single-phase images as input, and the change detection results are predicted to obtain the change detection results of the single-phase semantic feature decoder branch. In summary, the calculation process of this branch can be expressed by formula (3): Specifically, The subscripts 1 and 2 in the table refer to the phases, 1 represents the anterior phase and 2 represents the posterior phase. It represents the feature map of the remote sensing image before and after the output of the change detection adapter after adjustment, M θ (·) represents the single-temporal semantic feature decoder branch.

6. A dual-branch change detection method based on a visual base model as claimed in claim 5, characterized in that: The cross entropy loss is used to predict the results For supervision, the loss function is calculated as shown in formula (4): Specifically, N is the number of samples, C is the number of categories, and y i,c represents the indicator function of the true label of the i-th sample under category c. If it is in this category, it is 1, otherwise it is 0, and p i,c Then it is the predicted probability of the i-th sample under category c.

7. The dual-branch change detection method based on a visual base model as claimed in claim 1, characterized in that: The specific structure of the dual-phase change feature decoder is as follows: 51) The dual-phase change feature decoder includes branches and several decoding units, each of which consists of two convolution layers with a convolution kernel size of 3×3, two batch normalization layers BN and two ReLU activation functions; in the process of decoding the dual-phase change features, each decoding unit concatenates and fuses the extracted features of the same scale and the features after upsampling a low-scale feature, where the features of the same scale come from the outputs of the previous and next phase features and other decoding units, and the low-scale features come from the outputs of the later phase features and other decoding units; the fused feature map is further extracted through the convolution block to improve the feature expression ability of the network; after the decoding operation of all decoding units, 4 features R containing shallow and deep information are finally output. 1,1 , R 1,2 , R 1,3 and R 1,4 , the decoding process is shown in formula (5): Specifically, R i,j represents the output of the corresponding position decoding unit, F i Indicates the features from the previous phase, F′ i Represents features from the later phase, Upsample represents the upsampling operation, and Conv represents the convolution layer with a convolution kernel size of 3×3; 52) Next, the obtained features are input into a change detection module consisting of a channel attention module CAM, a convolutional layer and a sigmoid function; wherein CAM obtains the global information of each channel through global average pooling and global maximum pooling, then generates channel weights using a fully connected layer, and finally applies these weights to the original feature map, thereby achieving adaptive adjustment of the features; the change detection module receives the concatenated features of the four features output by the dual-phase change feature decoder in the channel dimension as input, and outputs the change detection result of the dual-phase change feature decoder branch In summary, the calculation process of this branch can be expressed by formula (6) as follows: Specifically, The subscripts 1 and 2 in the table refer to the phases, 1 represents the anterior phase and 2 represents the posterior phase. The feature map representing the before and after phase remote sensing images output by the change detection adapter after adjustment. Represents the dual-phase-change feature decoder branch.

8. The dual-branch change detection method based on a visual base model as claimed in claim 1, characterized in that: The calculation formula for weighted fusion of the change detection results output by the dual-branch change detection module is as follows: Specifically, is the change detection result of the single-phase semantic feature decoder branch, is the change detection result of the dual-phase change feature decoder branch, Conv 1×1 (·) represents 1×1 convolution operation, and Cat(·) represents channel concatenation.

9. A dual-branch change detection method based on a visual base model as claimed in claim 7 or 8, characterized in that: During the training process, cross entropy loss is used to predict the results. or To supervise.

10. A dual-branch change detection system based on a visual base model, characterized in that: It includes a processor and a memory, the memory is used to store program instructions, and the processor is used to call the stored instructions in the memory to execute a dual-branch change detection method based on a visual basic model as described in any one of claims 1 to 9.

Images