Semi-supervised remote sensing image change detection method based on pseudo bi-temporal generation technology

Abstract

The present application relates to a semi-supervised remote sensing image change detection method based on pseudo-bi-temporal generation technology. The method comprises: processing a small amount of single-phase remote sensing image data and building extraction labels thereof by pseudo-bi-temporal data generation technology, generating a large amount of pseudo-bi-temporal remote sensing images and pseudo change detection labels to provide label data support for the training of the change detection network; using a semi-supervised method, increasing or decreasing the color difference between the features of the bi-temporal remote sensing image pair by color enhancement in the strong enhancement process, deforming the ground objects by shape enhancement in the strong enhancement process, forcing the change detection network model to make consistent change detection results for the unlabeled remote sensing images before and after the strong enhancement, effectively utilizing a small amount of labeled single-phase remote sensing image pairs and a large number of unlabeled remote sensing image pairs to improve the change detection accuracy of remote sensing images, overcoming the problem of insufficient labeled data, and releasing the potential of remote sensing big data.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a semi-supervised remote sensing image change detection method based on pseudo dual-phase generation technology. Background Art

[0002] Remote sensing image change detection (CD) aims to identify changes that occur in remote sensing imagery acquired over time within the same area. Changes here generally refer to semantic changes. For many years, remote sensing image change detection has been a research hotspot in the remote sensing field. With the growing number of remote sensing images, improved image capture accuracy, and the advancement of deep learning technology, remote sensing image change detection technology can rapidly capture information about changes in areas of interest, including changes in natural features and man-made structures, providing strong support for decision-making by governments, companies, and organizations. To date, remote sensing image change detection technology has been widely applied in areas such as ecosystem monitoring, land resource and land use mapping, damage assessment, and urban expansion monitoring.

[0003] Currently, mainstream CD algorithms are based on fully supervised deep learning methods. These methods require a large number of labeled bi-temporal remote sensing image pairs for neural network training. Labeling these images is labor-intensive and time-consuming. Using unsupervised learning methods to train models often results in low change detection accuracy due to a lack of labeled data, making it insufficient for practical change detection applications.

[0004] In recent years, researchers have also begun studying CD methods based on semi-supervised learning (SSL). The goal is to improve the accuracy of CD tasks by training models with a small amount of labeled data and a large amount of unlabeled data. Training the model with a small amount of labeled data can guide the model to understand the task, while the large amount of unlabeled data can effectively prevent the model from overfitting on the small amount of labeled data, thereby improving the model's robustness and generalization. Currently, there are relatively few CD models based on SSL. Some researchers have implemented semi-supervised change detection using generative adversarial networks (GANs). By using a discriminator to enforce the consistency of feature distributions between labeled and unlabeled data, they can improve model performance by utilizing large amounts of unlabeled data. However, GAN networks are difficult to train, and this method requires labels that are variable, which places restrictions on labels and the amount of label data required, making it difficult to implement in practical applications. Summary of the Invention

[0005] Based on this, it is necessary to provide a semi-supervised remote sensing image change detection method based on pseudo dual-phase generation technology to address the above technical problems.

[0006] A semi-supervised remote sensing image change detection method based on pseudo bi-temporal generation technology, the method comprising:

[0007] Obtain labeled single-temporal remote sensing images and their building extraction labels, as well as unlabeled remote sensing image pairs;

[0008] The single-phase remote sensing image and its building extraction labels are processed according to the pseudo-bi-temporal generation technology to generate a pseudo-bi-temporal remote sensing image and its corresponding building extraction labels. The pseudo-bi-temporal remote sensing image is input into a pre-built change detection network for image change detection, and a supervised change detection result is output. The building extraction labels corresponding to the pseudo-bi-temporal remote sensing image are input into a pre-built change detection label generator for processing to generate pseudo change detection labels. The supervised change detection result and the pseudo change detection labels are calculated to obtain a supervised loss.

[0009] By performing weak enhancement processing on the unlabeled remote sensing image pair, and performing time exchange on the generated weakly enhanced unlabeled remote sensing image pair, generating the weakly enhanced unlabeled remote sensing image pair after time exchange, performing image change detection, strong enhancement processing and confidence filtering on the weakly enhanced unlabeled remote sensing image pair and the weakly enhanced unlabeled remote sensing image pair after time exchange respectively, generating pseudo labels and pseudo labels after time exchange, performing strong enhancement processing and image change detection on the weakly enhanced unlabeled remote sensing image pair and the weakly enhanced unlabeled remote sensing image pair after time exchange respectively, obtaining unsupervised change detection results and unsupervised change detection results after time exchange, and calculating according to the pseudo labels, the pseudo labels after time exchange, the unsupervised change detection results and the unsupervised change detection results after time exchange, to obtain the unsupervised loss;

[0010] The total loss is obtained by weighted calculation of supervised loss and unsupervised loss. The minimum total loss is used as the objective function to train and optimize the parameters in the change detection network until a trained change detection network is obtained. The remote sensing image to be detected is input into the trained change detection network for detection, and the change detection image is output.

[0011] In one embodiment, the pseudo-bi-phase generation technique includes a random superposition module and a time swap module;

[0012] The single-phase remote sensing image and its building extraction labels are processed according to the pseudo-bi-phase generation technology to generate pseudo-bi-phase remote sensing images and their corresponding building extraction labels, including:

[0013] According to the random overlay module, the single-phase remote sensing image and its building extraction labels are randomly sampled and randomly weakly enhanced in turn to generate pseudo-bi-phase remote sensing images and their corresponding building extraction labels after random overlay;

[0014] The pseudo bi-temporal remote sensing images after random superposition are time-exchanged according to the time-exchange module to generate pseudo bi-temporal remote sensing images after time-exchange.

[0015] In one embodiment, a random overlay module is used to perform random sampling and random weak enhancement processing on a single-temporal remote sensing image and its building extraction labels, thereby generating a pseudo dual-temporal remote sensing image and its corresponding building extraction labels after random overlay, including:

[0016] According to the random overlay module, the single-phase remote sensing image X st and its building extraction label Y be Perform two repeated random samplings to generate the first image sampling results X RS_A And the corresponding first label sampling result Y RS_A , and the second image sampling result X RS_B And the corresponding second label sampling result Y RS_B ;

[0017] The first image sampling result X RS_A And the corresponding first label sampling result Y RS_A Perform random weak enhancement processing to generate the first image sampling result X after random weak enhancement processing p1_A And the corresponding first label sampling result Y be_1 , the second image sampling result X RS_B And the corresponding second label sampling result Y RS_B Perform random weak enhancement processing to generate the second image sampling result X after random weak enhancement processing p1_B And the corresponding second label sampling result Y be_2 ;

[0018] Pair the image sampling results after random weak enhancement processing with the label sampling results to obtain pseudo dual-temporal remote sensing images after random superposition, including X p1_A and X p1_B , and the building extraction labels corresponding to the pseudo-bi-temporal remote sensing images after random superposition, including Y be_1 and Y be_2 .

[0019] In one embodiment, the pseudo dual-temporal remote sensing images after random superposition are time-exchanged according to the time-exchange module to generate the pseudo dual-temporal remote sensing images after time-exchange, including

[0020] According to the time exchange module, the first image sampling results X in the pseudo dual-temporal remote sensing images after random superposition are respectively p1_A And the second image sampling result X p1_B Perform time exchange to generate pseudo dual-phase remote sensing images after time exchange, including the second image sampling result X after time exchange p2_A The second image sampling result after time exchange X p2_B .

[0021] In one embodiment, the pseudo bi-temporal remote sensing image is input into a pre-built change detection network to perform image change detection, and the supervised change detection result is output, including:

[0022] The randomly superimposed pseudo-bi-temporal remote sensing image is input into the change detection network for image change detection, and the first type of supervised change detection result is output.

[0023] The pseudo-bi-temporal remote sensing image after time exchange is input into the change detection network for image change detection, and the second type of supervised change detection results are output.

[0024] In one embodiment, the building extraction labels corresponding to the pseudo dual-temporal remote sensing image are input into a pre-built change detection label generator for processing to generate the pseudo change detection labels, including:

[0025] According to the change detection label generator, the building extraction labels corresponding to the randomly superimposed pseudo dual-temporal remote sensing images are XORed to generate the pseudo change detection label Y l .

[0026] In one embodiment, a supervised loss is calculated based on the supervised change detection results and the pseudo change detection labels, including:

[0027] According to the first type of supervised change detection results and pseudo change detection label Y l Calculate and get the first type of supervised loss L st1 , according to the second type of supervised change detection results and pseudo change detection label Y l Calculate and get the second type of supervised loss L st2 ;

[0028] The first type of supervised loss L st1 and the second type of supervised loss L st2 Add together to get the supervised loss, expressed as

[0029] L s =L st1 +Lst2

[0030]

[0031]

[0032] Among them, L ce represents the cross entropy loss, L dice Indicates dice loss.

[0033] In one embodiment,

[0034] By performing weak enhancement processing on the unlabeled remote sensing image pair, and performing time exchange on the generated weakly enhanced unlabeled remote sensing image pair, generating the weakly enhanced unlabeled remote sensing image pair after time exchange, performing image change detection, strong enhancement processing and confidence filtering on the weakly enhanced unlabeled remote sensing image pair and the weakly enhanced unlabeled remote sensing image pair after time exchange respectively, generating pseudo labels and pseudo labels after time exchange, performing strong enhancement processing and image change detection on the weakly enhanced unlabeled remote sensing image pair and the weakly enhanced unlabeled remote sensing image pair after time exchange respectively, obtaining unsupervised change detection results and unsupervised change detection results after time exchange, and calculating according to the pseudo labels, the pseudo labels after time exchange, the unsupervised change detection results and the unsupervised change detection results after time exchange, the unsupervised loss is obtained, including:

[0035] By using unlabeled remote sensing images to u Perform weak enhancement processing and generate weak enhancement processing unlabeled remote sensing image pair X w Perform time exchange to generate weakly enhanced unlabeled remote sensing image pairs X after time exchange w_swap ;

[0036] X w and Input the change detection network to perform image change detection, and output the unsupervised change detection result with weak enhancement processing. Unsupervised change detection results after weak enhancement and time exchange Respectively and Perform strong enhancement processing and confidence filtering in sequence to generate pseudo labels Pseudo-label after time exchange

[0037] X w and X w_swap Perform strong enhancement processing to generate distorted image X s The distorted image X after time exchange s_swap , X s and X s_swapInput the change detection network to perform image change detection, and output the unsupervised change detection results. Unsupervised change detection results after time swapping

[0038] according to and Calculate and get the first type of unsupervised loss L ut1 ,according to and Calculate and get the second type of unsupervised loss L ut2 , L ut1 and L ut2 Add together to get the unsupervised loss L u .

[0039] In one embodiment, the total loss is obtained by weighted calculation of the supervised loss and the unsupervised loss, including:

[0040] The total loss is expressed as

[0041] L CD =L s +λL u

[0042] Where λ represents the weighting coefficient.

[0043] In one embodiment, the change detection network includes a twin encoder, an attention unit, and a decoder. The specific steps of the change detection network performing image change detection on an input image include:

[0044] The input image of the change detection network is downsampled according to the twin encoder with shared weights and parameters, the image features of the input image are extracted, and the image features are input into the attention unit to obtain the attention features integrated into the attention mechanism. The attention features are extracted and processed according to the decoder, and the change detection results are output.

[0045] In one embodiment, the weak enhancement processing includes translating and / or flipping the image or label by a preset amplitude.

[0046] In one embodiment, the strong enhancement processing includes performing color enhancement and / or shape enhancement of a preset intensity and a preset combination on the image or change detection result.

[0047] This semi-supervised remote sensing image change detection method, based on pseudo-bitemporal generation technology, specifically addresses the issues of a lack of labels for remote sensing image change detection and the labor-intensive and time-consuming labeling process. The pseudo-bitemporal data generation technique is introduced to the remote sensing image change detection problem. Using a small amount of single-phase remote sensing image data and its extracted building labels, a large number of pseudo-bitemporal remote sensing images and pseudo change detection labels are generated to provide labeled data support for training the change detection network model. Furthermore, a semi-supervised method based on the consistency regularization principle is used. Color enhancement in the strong enhancement process increases or decreases the color difference between features in bitemporal remote sensing image pairs, while shape enhancement in the strong enhancement process deforms the ground objects. By forcing the change detection network model to produce consistent change detection results for unlabeled remote sensing images before and after strong enhancement, the method effectively utilizes a small number of labeled single-phase remote sensing image pairs and a large number of unlabeled remote sensing image pairs to improve the accuracy of remote sensing image change detection. This overcomes the problem of insufficient labeled data, unleashing the potential of remote sensing big data, and has promising application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 1 is a flow chart of a semi-supervised remote sensing image change detection method based on pseudo dual-temporal generation technology in one embodiment;

[0049] Figure 2 A schematic diagram of a flow chart of a pseudo dual-phase generation technology in one embodiment;

[0050] Figure 3 A schematic diagram of a change detection network model in one embodiment;

[0051] Figure 4 A visual representation of an enhancement method in one embodiment;

[0052] Figure 5 A visual representation of a randomly combined strong enhancement method in one embodiment;

[0053] Figure 6 A visual representation of a confidence threshold filter in one embodiment;

[0054] Figure 7 FIG. 1 is a schematic diagram of test results of performing change detection on a trained change detection network by inputting a remote sensing image to be detected in one embodiment. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0056] In one embodiment, Figure 1 As shown in FIG, a semi-supervised remote sensing image change detection method based on pseudo dual-temporal generation technology is provided, and the specific steps include:

[0057] Obtain a small amount of labeled single-temporal remote sensing images and their building extraction labels, as well as a large number of unlabeled remote sensing image pairs;

[0058] The single-phase remote sensing image and its building extraction labels are processed according to the pseudo-bi-temporal generation technology to generate a pseudo-bi-temporal remote sensing image and its corresponding building extraction labels. The pseudo-bi-temporal remote sensing image is input into a pre-built change detection network for image change detection, and a supervised change detection result is output. The building extraction labels corresponding to the pseudo-bi-temporal remote sensing image are input into a pre-built change detection label generator for processing to generate pseudo change detection labels. The supervised change detection result and the pseudo change detection labels are calculated to obtain a supervised loss.

[0059] By performing weak enhancement processing on the unlabeled remote sensing image pair, and performing time exchange on the generated weakly enhanced unlabeled remote sensing image pair, generating the weakly enhanced unlabeled remote sensing image pair after time exchange, performing image change detection, strong enhancement processing and confidence filtering on the weakly enhanced unlabeled remote sensing image pair and the weakly enhanced unlabeled remote sensing image pair after time exchange respectively, generating pseudo labels and pseudo labels after time exchange, performing strong enhancement processing and image change detection on the weakly enhanced unlabeled remote sensing image pair and the weakly enhanced unlabeled remote sensing image pair after time exchange respectively, obtaining unsupervised change detection results and unsupervised change detection results after time exchange, and calculating according to the pseudo labels, the pseudo labels after time exchange, the unsupervised change detection results and the unsupervised change detection results after time exchange, to obtain the unsupervised loss;

[0060] The total loss is obtained by weighted calculation of supervised loss and unsupervised loss. The minimum total loss is used as the objective function to train and optimize the parameters in the change detection network until a trained change detection network is obtained. The remote sensing image to be detected is input into the trained change detection network for detection, and the change detection image is output.

[0061] In one embodiment, a single-temporal remote sensing image and its building extraction labels are processed according to a pseudo-bi-temporal generation technology to generate a pseudo-bi-temporal remote sensing image and its corresponding building extraction labels, including:

[0062] Among them, the Pseudo Bi-Temporal Data Generator (PBDG) technology aims to use a small number of single-phase remote sensing images with building extraction labels, expand them into a large number of pseudo bi-phase remote sensing image sets in terms of time and space dimensions, and generate their pseudo change detection labels at the same time. Figure 2 As shown in Figure 3, it consists of two parts: a random stacking module and a time exchange module.

[0063] a. The processing steps of the random stacking module include:

[0064] According to the random overlay module, the single-phase remote sensing image X st and its building extraction label Y be Perform two repeated random samplings to generate the first image sampling results X RS_A And the corresponding first label sampling result Y RS_A , and the second image sampling result X RS_B And the corresponding second label sampling result Y RS_B ;

[0065] The first image sampling result X RS_A And the corresponding first label sampling result Y RS_A Perform random weak enhancement processing to generate the first image sampling result X after random weak enhancement processing p1_A And the corresponding first label sampling result Y be_1 , the second image sampling result X RS_B And the corresponding second label sampling result Y RS_B Perform random weak enhancement processing to generate the second image sampling result X after random weak enhancement processing p1_B And the corresponding second label sampling result Y be_2 ;

[0066] Pair the image sampling results after random weak enhancement processing with the label sampling results to obtain pseudo dual-temporal remote sensing images after random superposition, including X p1_A and X p1_B , and the building extraction labels corresponding to the pseudo-bi-temporal remote sensing images after random superposition, including Y be_1 and Y be_2 .

[0067] It can be understood that the random overlay module is a key means to obtain a large number of pseudo bi-temporal remote sensing images from a small number of single-temporal remote sensing images in the pseudo bi-temporal data generation technology. Through random repeatable sampling and weak enhancement, the random overlay module makes the input single-temporal remote sensing image X stDifferent positions between the same or different images can be randomly superimposed on one piece to compare the semantic differences, which is equivalent to the semantic differences between any positions of any image can be utilized, thereby providing a far greater number of pseudo dual-temporal remote sensing images than the input single-temporal remote sensing images for the training of the change detection network model.

[0068] Assume that there are N input remote sensing images. If they are randomly paired without weak enhancement, and the image itself is paired with itself, the image can be paired with N images. Pseudo-bi-temporal remote sensing images. Suppose a pair of paired remote sensing images are subjected to random weak enhancement, including random flipping and / or translation of a set amplitude, and the two images use different weak enhancement techniques. Since the weak enhancement is completely random, the pixel areas where the two images are paired are also completely random. Then, there are as many new pseudo-bi-temporal remote sensing images as there are random weak enhancements. Assume that this number is M (M will be a very large number). Then, random sampling and random weak enhancement are applied simultaneously to the input single-phase remote sensing image X. st In theory, it can produce This method directly makes it possible to expand the original dataset of N single-phase remote sensing images into a dataset with This is impossible to achieve with the traditional method of using dual-temporal remote sensing images as training sets for network training. Only single-temporal remote sensing images that can be freely paired can achieve a large-scale expansion of this small amount of labeled data.

[0069] It should be noted that the random sampling used in the training process is replacement sampling, that is, repeated images are allowed. However, since the weak enhancement used for each image is also random, the possibility that the pseudo-bi-temporal remote sensing image is composed of two identical images in each training is very small and can be ignored.

[0070] b. The processing steps of the time exchange module include:

[0071] Based on the principle of time symmetry, the first image sampling result X in the pseudo dual-temporal remote sensing image after random superposition is respectively processed according to the time exchange module. p1_A And the second image sampling result X p1_B Perform time exchange to generate pseudo dual-phase remote sensing images after time exchange, including the second image sampling result X after time exchange p2_A The second image sampling result after time exchange X p2_B .

[0072] As you can understand, temporal symmetry refers to the fact that while changes between two dual-temporal remote sensing images have a temporal direction in reality, they are not directional for the change detection network. This means that for images T1 and T2, the change detection task is simply to identify the semantic information of the changes and classify the pixels. The change detection network model is unconcerned with whether the change actually occurred from image T1 to image T2 or vice versa. Temporal symmetry imposes a constraint on the training of the change detection network model: switching the input order of the two images should not affect the change detection results.

[0073] In one embodiment, the pseudo bi-temporal remote sensing image is input into a pre-built change detection network to perform image change detection, and the supervised change detection result is output, including:

[0074] The pseudo dual-temporal remote sensing images (including X p1_A and X p1_B ) Input the change detection network to perform image change detection and output the first type of supervised change detection results

[0075] The pseudo dual-temporal remote sensing image after time exchange (including X p2_A and X p2_B ) Input the change detection network to perform image change detection and output the second type of supervised change detection results

[0076] Among them, the change detection network model is as follows Figure 3 As shown in the figure, including the twin encoder, attention unit and decoder, the specific steps of the change detection network for image change detection on the input image include:

[0077] The input image of the change detection network is downsampled according to the twin encoder with shared weights and parameters, the image features of the input image are extracted, and the image features are input into the attention unit to obtain the attention features integrated into the attention mechanism. The attention features are extracted and processed according to the decoder, and the change detection results are output.

[0078] It can be understood that the twin encoder and decoder in the change detection network model are connected through skip connections, which can better integrate the deep semantic information and shallow spatial information of the image. The change detection network model aims to identify the semantic changes in remote sensing images acquired at different phases in the same area to obtain change detection results, where the change detection results are grayscale images of remote sensing image pairs with matching sizes.

[0079] In one embodiment, the building extraction labels corresponding to the pseudo dual-temporal remote sensing image are input into a pre-built change detection label generator for processing to generate the pseudo change detection labels, including:

[0080] According to the change detection label generator, the building labels corresponding to the randomly superimposed pseudo-bi-temporal remote sensing images (including Y be_1 and Y be_2 ) performs an XOR operation to generate a pseudo change detection label Y l .

[0081] It is understandable that since the label of the single-phase remote sensing image is only the building extraction label, after generating the paired pseudo-bi-phase remote sensing image, we also need to use the corresponding building extraction label to generate the pseudo change detection label. A and X B Obviously, their changing area should be the building in X A and X B Pixel positions that appear only once on . Assume that their building extraction labels are and They are all binary images, so by and By performing XOR operation, we can get A and X B The pixel position of the building that appears once is used as X A and X B Based on the above considerations, we use the XOR operation as the change detection label generator to obtain the pseudo change detection label Y l .

[0082] In one embodiment, a supervised loss is calculated based on the supervised change detection results and the pseudo change detection labels, including:

[0083] According to the first type of supervised change detection results and pseudo change detection label Y l Calculate and get the first type of supervised loss L st1 , according to the second type of supervised change detection results and pseudo change detection label Y l Calculate and get the second type of supervised loss L st2 ;

[0084] The first type of supervised loss L st1 and the second type of supervised loss L st2 Add together to get the supervised loss, expressed as

[0085] L s =L st1 +L st2

[0086]

[0087]

[0088] Among them, L ce represents the cross entropy loss, L dice Denotes the dice loss, respectively expressed as

[0089]

[0090]

[0091] Among them, Y represents the label, Indicates the change detection result, H is the height of the image, W is the width of the image, Represents the pixel value of the mth pixel of the image, c is 0 or 1, and c indicates whether the mth pixel of the image has changed or not.

[0092] It can be understood that the use of hybrid cross entropy loss function and dice loss function to calculate the loss function can reduce the imbalance effect caused by the fact that the unchanged pixels are much larger than the changed pixels in remote sensing image change detection.

[0093] Specifically, let ROM represent the random overlay module, Tswap represent the time swap module, CDLG represent the change detection label generator, P model is the change detection network model, θ is the model parameter, H is the loss function, and the calculation process of the above supervised loss can be described by the following formula:

[0094] (X p1_A , X p1_B , Y be_1 , Y be_2 )=ROM(X st , Y be )

[0095] (X p2_A , X p2_B )=Tswap(X p1_A , X p1_B )

[0096] Y l =CDLG(Y be_1 , Y be_2 )

[0097]

[0098]

[0099]

[0100] L s =L st1+L st2 .

[0101] After calculating the supervised loss using labeled data, we need to calculate the unsupervised loss using unlabeled data. The specific steps include:

[0102] First, we use the unlabeled remote sensing image X u Perform weak enhancement processing and generate weak enhancement processing unlabeled remote sensing image pair X w Perform time exchange to generate weakly enhanced unlabeled remote sensing image pairs X after time exchange w_swap , and X w and Input the change detection network to perform image change detection, and output the unsupervised change detection result with weak enhancement processing. Unsupervised change detection results after weak enhancement and time exchange Respectively and Perform strong enhancement processing and confidence filtering in sequence to generate pseudo labels Pseudo-label after time exchange

[0103] Then, X w and X w_swap Perform strong enhancement processing to generate distorted image X s The distorted image X after time exchange s_swap , X s and X s_swap Input the change detection network to perform image change detection, and output the unsupervised change detection results. Unsupervised change detection results after time swapping

[0104] Finally, according to and Calculate and get the first type of unsupervised loss L ut1 ,according to and Calculate and get the second type of unsupervised loss L ut2 , L ut1 and L ut2 Add together to get the unsupervised loss L u .

[0105] Specifically, let Augment w For weak enhancement, Augment s For strong enhancement, CTF is the confidence threshold filter, τ is the confidence threshold of the filter, and the process of calculating the unsupervised loss can be described by the following formula:

[0106] Xw =Augment w (X l )

[0107] X w_swap =Tswap(X w )

[0108]

[0109]

[0110]

[0111]

[0112]

[0113] L u =L ut1 +L ut2 .

[0114] Among them, strong enhancement processing is relative to weak enhancement, which refers to the enhancement that can cause huge distortion of remote sensing images, including color enhancement and / or shape enhancement of preset intensity and preset combination of images or change detection results. Figure 4 Specifically, as shown in Table 1, color enhancement and shape enhancement mainly include brightness change, color change, contrast change, equalization, tone separation, horizontal translation, rotation, sharpening, horizontal shearing, vertical shearing, exposure, vertical translation and cropping. In a specific embodiment, three methods other than cropping are randomly selected (with random intensity) and then combined with the cropping method to form a strong enhancement method. Figure 5 The results of some random combinations are listed.

[0115] Table 1 Enhancement methods and their intensity ranges and descriptions

[0116] Enhancement Method Intensity range illustrate brightness [0.05,0.95] Change the brightness of an image color [0.05,0.95] Change the color balance of an image Contrast [0.05,0.95] Change the contrast of an image Equalization / Perform histogram equalization on the image Posterization [4,8] Reduce the number of bits per color channel Rotation [-30,30] Rotate an image Sharpen [0.05,0.95] Adjust the sharpness of the image Transverse shear [-0.3,0.3] Shear the image along the horizontal axis Longitudinal shearing [-0.3,0.3] Shear the image along the vertical axis exposure [0,256] Invert all pixel values above a threshold Horizontal translation [-0.3,0.3] Pan in the horizontal direction Vertical translation [-0.3,0.3] Pan in the vertical direction cropping [0.25,0.35] Crop an area from the image

[0117] As you can understand, strong augmentation serves two main purposes. First, color augmentation increases or decreases the color difference between features in a pair of bi-temporal images. Second, shape augmentation deforms features. Consistency regularization requires the change detection network model to ignore these changes and focus on object changes, thereby improving the model's semantic understanding and robustness.

[0118] It is worth noting that for the same pair of images, the two strong enhancement processes need to be consistent, that is, Figure 1The two strong enhancements in should be consistent, so as to ensure that the pseudo labels correspond one-to-one with the pixels in the unsupervised change detection results.

[0119] It can be understood that confidence filtering refers to filtering according to the confidence threshold filter, which is used to filter out pixel prediction values (between 0 and 1) that exceed the confidence threshold τ, and pixel prediction values that are less than the confidence threshold τ will not participate in the unsupervised loss L u The confidence filtering results are as follows: Figure 6 This prevents low-confidence pixel predictions from being treated as pseudo labels and misleading the model.

[0120] In one embodiment, a total loss is obtained by weighted calculation of supervised loss and unsupervised loss, and the parameters in the change detection network are trained and optimized with minimizing the total loss as the objective function until a trained change detection network is obtained, including:

[0121] The total loss is obtained by weighted calculation of supervised loss and unsupervised loss, which is expressed as

[0122] L CD =L st1 +L st2 +λL u

[0123] Among them, λ represents the weighting coefficient;

[0124] Take the total loss L CD The minimum is the objective function, and the AdamW optimization algorithm is used to train and optimize the parameters in the change detection network. When the loss value no longer decreases, the network parameters at this time are saved as the final network model parameters to obtain the trained change detection network.

[0125] Furthermore, in order to verify the proposed semi-supervised remote sensing image change detection method based on pseudo dual-temporal generation technology, 50 pairs of labeled single-temporal remote sensing images and their building extraction labels, as well as 1950 pairs of unlabeled remote sensing images were input for training to obtain a trained change detection model. Figure 7 As shown in the figure, the first two columns are the input images, the third column is the extracted building labels corresponding to the single-temporal remote sensing images, the fourth column shows the change detection results of the model trained with 50 pairs of labeled single-temporal remote sensing images for full supervision, and the fifth column shows the change detection results of the model trained with 50 pairs of labeled single-temporal remote sensing images and 1950 pairs of unlabeled remote sensing images for semi-supervised training. This comparison shows that the proposed semi-supervised remote sensing image change detection method based on pseudo-bi-temporal generation technology produces remote sensing image change detection results that are closer to the true labels and have higher change detection accuracy.

[0126] It should be understood that although Figure 1 and Figure 2 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. In addition, Figure 1 and Figure 2 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0127] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0128] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A semi-supervised remote sensing image change detection method based on pseudo-bi-temporal generation technology, characterized in that: The method comprises: Obtain labeled single-temporal remote sensing images and their building extraction labels, as well as unlabeled remote sensing image pairs; The single-phase remote sensing image and its building extraction labels are processed according to a pseudo-bi-phase generation technology to generate a pseudo-bi-phase remote sensing image and its corresponding building extraction labels, the pseudo-bi-phase remote sensing image is input into a pre-built change detection network to perform image change detection, and a supervised change detection result is output, the building extraction labels corresponding to the pseudo-bi-phase remote sensing image are input into a pre-built change detection label generator for processing to generate pseudo change detection labels, and a supervised loss is calculated based on the supervised change detection result and the pseudo change detection labels. The method comprises the following steps: performing weak enhancement processing on the unlabeled remote sensing image pair, performing time exchange on the generated weak enhancement processed unlabeled remote sensing image pair, generating a time exchanged weak enhancement processed unlabeled remote sensing image pair, performing image change detection, strong enhancement processing and confidence filtering on the weak enhancement processed unlabeled remote sensing image pair and the time exchanged weak enhancement processed unlabeled remote sensing image pair respectively, generating a pseudo label and a time exchanged pseudo label, performing strong enhancement processing and image change detection on the weak enhancement processed unlabeled remote sensing image pair and the time exchanged weak enhancement processed unlabeled remote sensing image pair respectively, obtaining an unsupervised change detection result and an unsupervised change detection result after time exchange, and performing calculation according to the pseudo label, the time exchanged pseudo label, the unsupervised change detection result and the time exchanged unsupervised change detection result to obtain an unsupervised loss; The total loss is obtained by weighted calculation of the supervised loss and the unsupervised loss. The parameters in the change detection network are trained and optimized with the minimum total loss as the objective function until a trained change detection network is obtained. The remote sensing image to be detected is input into the trained change detection network for detection, and a change detection image is output.

2. The method according to claim 1, characterized in that The pseudo-bi-phase generation technology includes a random superposition module and a time exchange module; The single-temporal remote sensing image and its building extraction labels are processed according to a pseudo-bi-temporal generation technology to generate a pseudo-bi-temporal remote sensing image and its corresponding building extraction labels, including: According to the random overlay module, random sampling and random weak enhancement processing are sequentially performed on the single-temporal remote sensing image and the building extraction label thereof to generate a pseudo dual-temporal remote sensing image and its corresponding building extraction label after random overlay; The randomly superimposed pseudo dual-temporal remote sensing images are time-exchanged according to the time-exchange module to generate time-exchanged pseudo dual-temporal remote sensing images.

3. The method according to claim 2, characterized in that The random overlay module sequentially performs random sampling and random weak enhancement processing on the single-temporal remote sensing image and the building extraction label thereof to generate a pseudo dual-temporal remote sensing image and its corresponding building extraction label after random overlay, including: According to the random superposition module, the single-phase remote sensing image X st and its building extraction label Y be Perform two repeated random samplings to generate the first image sampling results X RS_A And the corresponding first label sampling result Y RS_A , and the second image sampling result X RS_B And the corresponding second label sampling result Y RS_B ; The first image sampling result X RS_A And the corresponding first label sampling result Y RS_A Perform random weak enhancement processing to generate the first image sampling result X after random weak enhancement processing p1_A And the corresponding first label sampling result Y be_1 , the second image sampling result X RS_B And the corresponding second label sampling result Y RS_B Perform random weak enhancement processing to generate the second image sampling result X after random weak enhancement processing p1_B And the corresponding second label sampling result Y be_2 ; Pair the image sampling results after random weak enhancement processing with the label sampling results to obtain pseudo dual-temporal remote sensing images after random superposition, including X p1_A and X p1_B , and the building extraction labels corresponding to the pseudo-bi-temporal remote sensing images after random superposition, including Y be_1 and Y be_2 .

4. The method according to claim 3, characterized in that The pseudo dual-temporal remote sensing images after random superposition are time-exchanged according to the time exchange module to generate pseudo dual-temporal remote sensing images after time exchange, including: According to the time exchange module, the first image sampling result X in the pseudo dual-temporal remote sensing image after random superposition is respectively p1_A And the second image sampling result X p1_B Perform time exchange to generate pseudo dual-phase remote sensing images after time exchange, including the second image sampling result X after time exchange p2_A The second image sampling result after time exchange X p2_B .

5. The method according to claim 1, wherein The pseudo bi-temporal remote sensing image is input into a pre-built change detection network to perform image change detection, and a supervised change detection result is output, including: The pseudo dual-temporal remote sensing images after random superposition are input into the change detection network to perform image change detection, and the first type of supervised change detection results are output. The pseudo-bi-temporal remote sensing image after time exchange is input into the change detection network to perform image change detection, and the second type of supervised change detection result is output.

6. The method according to claim 1, characterized in that The building extraction labels corresponding to the pseudo dual-temporal remote sensing image are input into a pre-built change detection label generator for processing to generate pseudo change detection labels, including: According to the change detection label generator, the building extraction labels corresponding to the randomly superimposed pseudo dual-temporal remote sensing images are subjected to an XOR operation to generate a pseudo change detection label Y l .

7. The method according to claim 1, characterized in that A supervised loss is obtained by calculating the supervised change detection result and the pseudo change detection label, including: According to the first type of supervised change detection results and pseudo change detection label Y l Calculate and get the first type of supervised loss L st1 , according to the second type of supervised change detection results and pseudo change detection label Y l Calculate and get the second type of supervised loss L st2 ; The first class has a supervised loss L st1 and the second type of supervised loss L st2 Add together to get the supervised loss, expressed as L s =L st1 +L st2 Among them, L ce represents the cross entropy loss, L dice Indicates dice loss.

8. The method according to claim 7, characterized in that The unlabeled remote sensing image pair is weakly enhanced, and the generated weakly enhanced unlabeled remote sensing image pair is time-swapped to generate a time-swapped weakly enhanced unlabeled remote sensing image pair. The weakly enhanced unlabeled remote sensing image pair and the time-swapped weakly enhanced unlabeled remote sensing image pair are respectively subjected to image change detection, strong enhancement processing, and confidence filtering in sequence to generate pseudo labels and time-swapped pseudo labels. The weakly enhanced unlabeled remote sensing image pair and the time-swapped weakly enhanced unlabeled remote sensing image pair are respectively subjected to strong enhancement processing and image change detection in sequence to obtain an unsupervised change detection result and an unsupervised change detection result after time swapping. The unsupervised loss is obtained according to the pseudo labels, the time-swapped pseudo labels, the unsupervised change detection result, and the time-swapped unsupervised change detection result. By using unlabeled remote sensing images to u Perform weak enhancement processing and generate weak enhancement processing unlabeled remote sensing image pair X w Perform time exchange to generate weakly enhanced unlabeled remote sensing image pairs X after time exchange w_swap ; X w and Input the change detection network to perform image change detection, and output the unsupervised change detection result with weak enhancement processing. Unsupervised change detection results after weak enhancement and time exchange Respectively and Perform strong enhancement processing and confidence filtering in sequence to generate pseudo labels Pseudo-label after time exchange For the X w and X w_swap Perform strong enhancement processing to generate distorted image X s The distorted image X after time exchange s_swap , X s and X s_swap Input the change detection network to perform image change detection, and output the unsupervised change detection result Unsupervised change detection results after time swapping according to and Calculate and get the first type of unsupervised loss L ut1 ,according to and Calculate and get the second type of unsupervised loss L ut2 , L ut1 and L ut2 Add together to get the unsupervised loss L u .

9. The method according to claim 8, characterized in that The total loss is obtained by weighting the supervised loss and the unsupervised loss, including: The total loss is expressed as THE CD =L s +λL u Where λ represents the weighting coefficient.

10. The method according to claim 1 or 5, characterized in that The change detection network includes a twin encoder, an attention unit, and a decoder. The specific steps of the change detection network performing image change detection on the input image include: The input image of the change detection network is downsampled according to the twin encoder with shared weights and parameters, the image features of the input image are extracted, the image features are input into the attention unit to obtain the attention features integrated into the attention mechanism, the attention features are extracted and processed according to the decoder, and the change detection results are output.

11. The method according to claim 1 or 2 or 3 or 8, characterized in that: The weak enhancement processing includes performing a preset amplitude translation and / or flipping on the image or label.

12. The method according to claim 1 or 8, characterized in that The strong enhancement processing includes performing color enhancement and / or shape enhancement with a preset intensity and a preset combination on the image or the change detection result.

Images