A semi-supervised remote sensing image change detection method and device based on multi-scale feature fusion

Abstract

The application provides a kind of semi-supervised remote sensing image change detection method and device based on multi-scale feature fusion, which comprises obtaining remote sensing image change detection data labeled training set and unlabeled training set, constructing teacher network model and student network model, training teacher network model using labeled training set, and initializing student network model parameters with trained teacher network model parameters, inputting student network model and teacher network model using unlabeled training set, obtaining respective prediction results;With the prediction result of the teacher network model as the pseudo-label, the student network model parameters are optimized, and the teacher network model parameters are updated, and the remote sensing image change detection model is finally obtained by iterative training;Remote sensing image change detection is carried out based on the detection model.The application can extract spatial information of different scales, enhance features, and effectively fuse feature information of different scales to improve the prediction performance of the model.

Description

Technical Field

[0001] This invention belongs to the field of remote sensing image processing technology, and in particular relates to a semi-supervised remote sensing image change detection method and device based on multi-scale feature fusion. Background Technology

[0002] In recent years, with the development of remote sensing satellite technology, the amount of remote sensing image data acquired has become increasingly abundant, making change detection using remote sensing images as a data source a research hotspot. Change detection refers to the process of identifying and analyzing changes in the Earth's surface or features over time and space using remote sensing technology or other related technologies in images or data acquired at different times or under different conditions.

[0003] In traditional remote sensing image change detection, acquiring high-quality labeled data is costly and time-consuming, especially for large-scale, long-term monitoring, where fully supervised methods are insufficient. Remote sensing images are inherently large, containing a significant number of targets per image. Manual labeling is extremely time-consuming, and due to limitations in the labeler's experience and skill level, the labeling results may differ from the actual changed areas, making it difficult to guarantee the accuracy of the labeled samples and affecting the precision and reliability of subsequent change detection. Summary of the Invention

[0004] To address the issues of low utilization of unlabeled data sample information and misleading pseudo-labels in the field of semi-supervised change detection, this invention aims to provide a semi-supervised remote sensing image change detection method and device based on multi-scale feature fusion. By adopting a student-teacher network structure, the teacher model generates pseudo-labels to assist the student model in training. Furthermore, a feature enhancement module and a multi-scale feature fusion module are used to capture effective information in unlabeled data, thereby achieving the goal of accurately extracting changed regions in change detection.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] In a first aspect, the present invention provides a semi-supervised remote sensing image change detection method based on multi-scale feature fusion, comprising:

[0007] Pre-construct a semi-supervised remote sensing image change detection and prediction model;

[0008] The remote sensing image to be detected is input into the semi-supervised remote sensing image change detection and prediction model to obtain the detection result of the remote sensing image change; the remote sensing image to be detected refers to a dual-temporal remote sensing image generated by taking pictures of the same area at different times, including the remote sensing image before the change and the remote sensing image after the change; the detection result refers to a black and white image of the changed area with the same size as the input remote sensing image, where black represents the background area that has not changed and white represents the area that has changed;

[0009] The pre-built semi-supervised remote sensing image change detection and prediction model includes:

[0010] Obtain a remote sensing image change detection dataset and divide it into a labeled training set and an unlabeled training set. The label refers to the pixel-level annotation of the change area between two bi-temporal images, with the change area being white and the no-change area being black.

[0011] A teacher network model and a student network model are constructed. Both the teacher network model and the student network model consist of a semantic feature extraction module, a multi-scale feature enhancement module, a multi-scale feature fusion module, and a prediction module. The teacher network model and the student network model are used to extract multi-scale semantic feature maps of the input remote sensing image and perform feature enhancement and interactive fusion to obtain the prediction result of the changes in the remote sensing image.

[0012] The teacher network model is trained using a labeled training set, and the parameters of the student network model are initialized with the parameters of the trained teacher network model. The student network model and the trained teacher network model are input using an unlabeled training set to obtain the prediction results of the teacher network model and the student network model respectively.

[0013] The prediction results of the teacher network model are used as pseudo-labels to optimize the parameters of the student network model and update the parameters of the teacher network model. The student network model is then trained and the teacher model parameters are updated using an unlabeled training set until training terminates. The optimal parameters of the student network model are saved as the remote sensing image change detection model.

[0014] Preferably, the semantic feature extraction module employs a deep learning network, and the semantic feature extraction module includes 5 feature extraction layers.

[0015] For the input remote sensing image before the change and the image after the change Feature extraction yields two feature maps. and ,use minus Semantic feature map as output of the feature extraction layer , .

[0016] Preferably, the multi-scale feature enhancement module is used to enhance the multi-level feature information of the semantic feature map, and the data processing is as follows:

[0017] right , Global average pooling, 1×1 convolution, ReLU function, 1×1 convolution, and Sigmoid function are performed to obtain the feature map. ;

[0018] right , Global average pooling, 3×3 convolution, ReLU function, 1×1 convolution, and Sigmoid function are performed to obtain the feature map. ;

[0019] right , Global average pooling, 5×5 convolution, ReLU function, 1×1 convolution, and Sigmoid function are performed to obtain the feature map. ;

[0020] Will , and The feature map is obtained by performing a stitching operation. ;

[0021] Will After global max pooling, 1×1 convolution, and ReLU operations, and... Adding them together yields the enhanced feature map. , .

[0022] Preferably, the multi-scale feature fusion module is used to interact the semantic features of remote sensing images at different scales to obtain a feature map of the changed region. The data processing is as follows:

[0023] right After upsampling, 1×1 convolution, ReLU and global average pooling, and The feature maps are obtained by performing addition operations followed by global average pooling. ,

[0024] right After upsampling, 1×1 convolution, ReLU and global average pooling, and The feature maps are obtained by performing addition operations followed by global average pooling. ,

[0025] right After upsampling, 1×1 convolution, ReLU and global average pooling, and The feature maps are obtained by performing addition operations followed by global average pooling. ,

[0026] right After upsampling, 1×1 convolution, ReLU and global average pooling, and The feature maps are obtained by performing addition operations followed by global average pooling. ,

[0027] Will Perform downsampling and 1×1 convolution, Perform downsampling and 1×1 convolution. Perform upsampling and 1×1 convolution. After upsampling and 1×1 convolution, and then with The feature map is obtained by performing a stitching operation. ,

[0028] Will The final feature map of the change region is obtained after a 1×1 convolution. .

[0029] Preferably, the prediction module is used to generate the feature map of the changed region. The image is upsampled to restore its original size, and a 1×1 convolution is used to output the predicted changes in the remote sensing image.

[0030] Preferably, the training process of the teacher network model is as follows:

[0031] Initialize the parameters of the teacher network model;

[0032] Select labeled training set , and These refer to the image before the remote sensing image changes and the image after the remote sensing image changes, respectively. It is tag information;

[0033] Will and The input is processed by the semantic feature extraction module in the teacher network model. and ,Will and Subtraction yields the feature map. Feature map Enhanced feature maps are obtained through the multi-scale feature enhancement module. The feature map of the changed region is obtained through the multi-scale feature fusion module. The prediction module outputs the detection results of image changes. ;

[0034] The loss is calculated based on the loss function, and the network model parameters are updated through backpropagation. The network model is trained iteratively until the maximum number of iterations is reached, and the optimal network model parameters are saved as the teacher model.

[0035] The loss function is:

[0036] ,

[0037] in, This represents the number of data points in the labeled training set.

[0038] Preferably, the training process of the student network model is as follows:

[0039] The parameters of the student network model are initialized using the parameters of the trained teacher model;

[0040] Select a set of unlabeled training data. ,

[0041] Will and The input is given to the student network model, and the semantic feature extraction module obtains the desired results. and ,Will and Subtraction yields the feature map. Feature map Enhanced feature maps are obtained through the multi-scale feature enhancement module. The feature map of the changed region is obtained through the multi-scale feature fusion module. The prediction module outputs the detection results of image changes. ; this group The data is simultaneously input into the trained teacher model to obtain the output results. As a pseudo-label;

[0042] The loss is calculated based on the pseudo-labels, and the parameters of the student network model are updated through backpropagation.

[0043] The teacher model parameters are updated using a moving exponential average, as follows:

[0044] ,in For teacher model parameters, For student network model parameters;

[0045] Continue iteratively training using the unlabeled training set in the manner described above until the maximum number of iterations is reached. Save the optimal student network model parameters and use the trained student network model as the remote sensing image change detection model.

[0046] Secondly, the present invention provides a semi-supervised remote sensing image change detection device based on multi-scale feature fusion, used to implement the above-mentioned semi-supervised remote sensing image change detection method based on multi-scale feature fusion, the device comprising:

[0047] The semi-supervised remote sensing image change detection and prediction model training module is used to train the semi-supervised remote sensing image change detection and prediction model.

[0048] The semi-supervised remote sensing image change detection and prediction model includes a teacher network model and a student network model. Both the teacher network model and the student network model consist of a semantic feature extraction module, a multi-scale feature enhancement module, a multi-scale feature fusion module, and a prediction module. The semantic feature extraction module extracts multi-scale semantic feature maps from the input remote sensing image. The multi-scale feature enhancement module enhances the multi-level feature information of the acquired semantic feature maps. The multi-scale feature fusion module interactively fuses semantic features of remote sensing images at different scales to obtain a change region feature map. The prediction module outputs the detection result of remote sensing image changes based on the change region feature map.

[0049] The training method is as follows: A remote sensing image change detection dataset is acquired and divided into a labeled training set and an unlabeled training set. The label refers to pixel-level annotation of the change region between two bi-temporal images, with the changed region marked in white and the unchanged region marked in black. A teacher network model and a student network model are constructed. The teacher network model is trained using the labeled training set, and the student network model parameters are initialized using the trained teacher network model parameters. The student network model and the trained teacher network model are input using the unlabeled training set to obtain the prediction results of the teacher network model and the student network model, respectively. The prediction results of the teacher network model are used as pseudo-labels to optimize the student network model parameters and update the teacher network model parameters. The student network model is trained and the teacher model parameters are updated using the unlabeled training set until training terminates. The optimal student network model parameters are saved as the remote sensing image change detection model.

[0050] It also includes: a detection output module, used to input the remote sensing image to be detected into the semi-supervised remote sensing image change detection and prediction model to obtain the detection result of the remote sensing image change; the remote sensing image to be detected refers to a dual-temporal remote sensing image generated by taking pictures of the same area at different times, including the remote sensing image before the change and the remote sensing image after the change; the detection result refers to a black and white image of the changed area with the same size as the input remote sensing image, where black represents the background area that has not changed and white represents the area that has changed.

[0051] Thirdly, the present invention provides a computer-readable storage medium storing one or more programs, the one or more programs including instructions that, when executed by a computing device, cause the computing device to perform any of the methods described above for semi-supervised remote sensing image change detection based on multi-scale feature fusion.

[0052] Fourthly, the present invention provides a computing device comprising one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising instructions for performing any of the methods described above for semi-supervised remote sensing image change detection methods based on multi-scale feature fusion.

[0053] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0054] This invention provides a semi-supervised remote sensing image change detection method based on multi-scale feature fusion, applicable to change detection tasks in scenarios with limited labeled data. This invention extracts spatial information at different scales and further augments the extracted feature maps, utilizing different receptive fields to acquire multi-scale feature information. The multi-scale feature fusion module effectively fuses feature information from different scales, thereby more accurately capturing the boundaries of changed regions in remote sensing images, generating accurate labeled results for truly changed regions, and improving the model's predictive performance. Attached Figure Description

[0055] Figure 1 This is a schematic diagram of the training process of the semi-supervised remote sensing image change detection model in an embodiment of the present invention;

[0056] Figure 2 This is a structural diagram of the semi-supervised remote sensing image change detection model provided in an embodiment of the present invention;

[0057] Figure 3 This is a schematic diagram of the data processing process of the feature enhancement module in an embodiment of the present invention;

[0058] Figure 4 This is a schematic diagram of the data processing process of the multi-scale feature fusion module in an embodiment of the present invention. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the embodiments and accompanying drawings. Here, the illustrative embodiments and descriptions of this invention are used to explain the invention, but are not intended to limit the invention.

[0060] It should also be noted that, in order to avoid obscuring the invention with unnecessary details, only the structures and / or processing steps closely related to the solution according to the invention are shown in the accompanying drawings, while other details that are not closely related to the invention are omitted.

[0061] It should be emphasized that the term "including / comprises" as used herein refers to the presence of a feature, element, step, or component, but does not exclude the presence or addition of one or more other features, elements, steps, or components.

[0062] In the following description, embodiments of the invention will be illustrated with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar parts, or the same or similar steps.

[0063] It is important to emphasize that the step markers mentioned below are not intended to limit the order of the steps. Rather, they should be understood as meaning that the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0064] The first embodiment of this invention provides a semi-supervised remote sensing image change detection method based on multi-scale feature fusion, comprising:

[0065] Pre-construct a semi-supervised remote sensing image change detection and prediction model;

[0066] The remote sensing image to be detected is input into the semi-supervised remote sensing image change detection and prediction model to obtain the detection result of the remote sensing image change. The remote sensing image to be detected is a dual-temporal remote sensing image generated by a UAV or satellite taking pictures of the same area at different times. The first image is the remote sensing image before the change, and the second image is the remote sensing image after the change. The detection result is a black and white image of the changed area with the same size as the input remote sensing image. Black represents the background area that has not changed, and white represents the area that has changed between the two input remote sensing images.

[0067] Among them, a semi-supervised remote sensing image change detection and prediction model is pre-built, see [link / reference]. Figure 1 The specific implementation process is as follows:

[0068] 1) Obtain the remote sensing image change detection dataset and divide it into a labeled training set and an unlabeled training set. The label refers to the pixel-level annotation of the change area between two bi-temporal images, with the change area in white and the unchange area in black.

[0069] 2) Constructing a teacher network model With student network model ,

[0070] See Figure 2Teacher network model With student network model Each module consists of a semantic feature extraction module, a multi-scale feature enhancement module, a multi-scale feature fusion module, and a prediction module; the teacher network model. With student network model This is used to extract semantic feature maps from input remote sensing images, enhance the multi-level feature information of the acquired semantic feature maps, and interactively fuse the semantic features of remote sensing images of different scale change areas to obtain change area feature maps and output the final prediction results.

[0071] 3) Training the teacher network model ,

[0072] Labeled training set The network model is trained by inputting the teacher network model, and the network parameters are updated through backpropagation. The optimal parameters are saved as the teacher model.

[0073] 4) Train the student network model ,

[0074] The training set will be processed by the labeled training set. The parameters of the trained teacher model are used as the student network model. The initialization parameters will be used to initialize the unlabeled training set. Simultaneously input the student network model Combined with the trained teacher model, we obtain the prediction results of the teacher model and the student network model, respectively.

[0075] The prediction results of the teacher model are then used as pseudo-labels and input into the student network model for further optimization and training. In each iteration, the parameters of the optimized student network model are synchronized to the teacher model through a moving exponential average. The student network model is then trained and the teacher model parameters are updated using an unlabeled training set until training terminates. The optimal parameters of the student network model are saved, and the trained student network model is used as a remote sensing image change detection model.

[0076] 5) Input the test set into the remote sensing image change detection model, and verify the performance of the remote sensing image change detection model by the difference between the label data in the test set and the model prediction results.

[0077] In this embodiment, the Levir-CD dataset is used as the dataset for remote sensing image change detection.

[0078] It should be noted that during training, the network parameters are updated through backpropagation to minimize the loss function. A maximum number of iterations is set, and when the maximum number of iterations is reached, the model parameters at this point are saved as the optimal network parameters and imported into the network model as a remote sensing image change detection and prediction model.

[0079] It should be noted that, in one embodiment, the maximum number of iterations for the teacher network model is 100, and the maximum number of iterations for the student network model is 50.

[0080] It should be noted that the moving exponential average algorithm used is employed to pass the parameters of the student network model to the teacher network model, specifically including:

[0081] The parameters of the teacher network model are The parameters of the student network model are ,but,

[0082] ,

[0083] In this embodiment, The value is 0.95.

[0084] In one embodiment, a remote sensing image change detection dataset is obtained and divided into a labeled training set and an unlabeled training set, as follows:

[0085] 11) Obtain the remote sensing image change detection dataset ,Will The training set was divided in a 5:5 ratio. and test set ;

[0086] 12) Transfer the training set Select 10% of the data sample The labeled training set is used as the training set, and the remaining training set is used as the unlabeled training set. ,in , and These refer to the image before the remote sensing image changes and the image after the remote sensing image changes, respectively. (Label information) It involves pixel-level annotation of the changed regions between two bi-temporal images, with changed regions marked in white and unchanged regions marked in black.

[0087] In one embodiment, the semantic feature extraction module extracts semantic features from the input image based on different receptive fields using convolution operations, batch regularization, ReLU function, and max pooling operations. The specific process is as follows:

[0088] 21) Suppose the input remote sensing image is the image before the change. and the image after the change Input size is ,in In this embodiment, the middle 224, Also 224, representing the height and width of the image;

[0089] 22) Input the preprocessed remote sensing image into the semantic feature extraction module. In this embodiment, the semantic feature extraction module is a ResNet-50 network, but it can also be other types of feature extraction networks such as VGG16, DeeplabV3, and other deep learning networks to obtain two feature maps for each feature extraction layer. and ,use minus As the output feature of this feature extraction layer ;

[0090] 23) The semantic feature extraction module network structure consists of 5 layers, and the feature maps that can be extracted from each layer are as follows: ;in , , The first Each feature extraction layer outputs the height, width, and number of channels of the feature map.

[0091] In one embodiment, the multi-scale feature enhancement module is used to enhance the acquisition of multi-level feature information from the semantic feature map, which helps to accurately extract blurred boundaries and improve model performance. See also Figure 3 The data processing procedure of the multi-scale feature enhancement module is as follows:

[0092] Feature maps of each layer Perform convolution operations with kernels of different sizes, including:

[0093] 31) To Global average pooling, 1×1 convolution, ReLU function, 1×1 convolution, and Sigmoid function are performed to obtain the feature map. , represented as ;

[0094] 32) To Global average pooling, 3×3 convolution, ReLU function, 1×1 convolution, and Sigmoid function are performed to obtain the feature map. , represented as ;

[0095] 33) To Global average pooling, 5×5 convolution, ReLU function, 1×1 convolution, and Sigmoid function are performed to obtain the feature map. , represented as ;in Indicates global average pooling. This represents a 5×5 convolution. Represents the ReLU function. Represents a 1×1 convolution. Represents the Sigmoid function;

[0096] 34) will , and The feature map is obtained by performing a stitching operation. , represented as: ,in This indicates a splicing operation.

[0097] Then After global max pooling, 1×1 convolution, and ReLU operations, and... The feature map after enhancement is obtained by adding the features together. , represented as:

[0098] ,

[0099] in Indicates global max pooling. Represents the ReLU function. Represents a 1×1 convolution. It indicates addition.

[0100] In one embodiment, a multi-scale feature fusion module is used for the interaction of semantic features of remote sensing images of regions with changes at different scales, to obtain a feature map of the changed regions. See also Figure 4 The data processing procedure of the multi-scale feature fusion module is as follows:

[0101] 41) To After upsampling, 1×1 convolution, ReLU and global average pooling, and The feature maps are obtained by performing addition operations followed by global average pooling. , represented as:

[0102] ;

[0103] 42) To After upsampling, 1×1 convolution, ReLU and global average pooling, and The feature maps are obtained by performing addition operations followed by global average pooling. , represented as:

[0104] ;

[0105] 43) To After upsampling, 1×1 convolution, ReLU and global average pooling, and The feature maps are obtained by performing addition operations followed by global average pooling. , represented as:

[0106] ;

[0107] 44) To After upsampling, 1×1 convolution, ReLU and global average pooling, and The feature maps are obtained by performing addition operations followed by global average pooling. , represented as:

[0108] ,

[0109] in Indicates an upsampling operation;

[0110] 45) Perform downsampling and 1×1 convolution, Perform downsampling and 1×1 convolution. Perform upsampling and 1×1 convolution. After upsampling and 1×1 convolution, and then with The feature map is obtained by performing a stitching operation. , represented as:

[0111] ,

[0112] in This indicates a downsampling operation.

[0113] Will The final feature map is obtained after a 1×1 convolution. , is represented as; .

[0114] In one embodiment, the prediction module upsamples... The feature map is restored to the size of the original image, and a 1×1 convolution is used to output the final prediction result. The prediction result is a black and white image of the changed area with the same size as the input remote sensing image, where black represents the background area that has not changed, and white represents the area that has changed between the two input remote sensing images.

[0115] In one embodiment, the training process of the teacher network model is as follows:

[0116] 51) Initialize the parameters of the teacher network model;

[0117] 52) Select a labeled training set. ,

[0118] 53) will and The input is processed by the semantic feature extraction module in the teacher network model. and ,Will and Subtraction yields the feature map. Feature map Enhanced feature maps are obtained through the multi-scale feature enhancement module. The feature map of the changed region is obtained through the multi-scale feature fusion module. ;Will The prediction module obtains the detection results of changes in the input image. ;

[0119] 54) Based on the loss function Calculate the loss and update the parameters through backpropagation;

[0120] 55) Iterate through the training process as described above until the maximum number of iterations is reached, and save the optimal parameters as the teacher model.

[0121] In one embodiment, the training process of the student network model is as follows:

[0122] 61) Use the parameters of the teacher model to initialize the parameters of the student network model;

[0123] 62) Select a set of unlabeled training data. ,

[0124] 63) will and The input is given to the student network model, and the semantic feature extraction module obtains the desired results. and ,Will and Subtraction yields the feature map. Feature map Enhanced feature maps are obtained through the multi-scale feature enhancement module. Feature maps are obtained through a multi-scale feature fusion module. ;Will The prediction module obtains the change detection result of this input. ; this group The data is simultaneously input into the teacher model to obtain the output results. As a pseudo-label;

[0125] 64) Based on the loss function Calculate the loss and update the student network model parameters through backpropagation;

[0126] 65) By moving exponential average Update the parameters of the teacher model, where the teacher model parameters are: The parameters of the student network model are ;

[0127] 66) Repeat steps 62)-65) for iterative training until the maximum number of iterations is reached, and save the optimal student network model parameters as the remote sensing image change detection model.

[0128] Based on the above-mentioned inventive concept, the present invention also provides a semi-supervised remote sensing image change detection device based on multi-scale feature fusion, used to implement the above-mentioned semi-supervised remote sensing image change detection method based on multi-scale feature fusion, the device comprising:

[0129] The semi-supervised remote sensing image change detection and prediction model training module is used to train a semi-supervised remote sensing image change detection and prediction model. This model includes a teacher network model and a student network model. See [link / reference]. Figure 2 Both the teacher network model and the student network model consist of a semantic feature extraction module, a multi-scale feature enhancement module, a multi-scale feature fusion module, and a prediction module. The semantic feature extraction module is used to extract multi-scale semantic feature maps from the input remote sensing image. The multi-scale feature enhancement module is used to enhance the multi-level feature information of the acquired semantic feature map. The multi-scale feature fusion module is used to interactively fuse the semantic features of remote sensing images in different scale change areas to obtain change area feature maps. The prediction module is used to output the final prediction result.

[0130] The training process of the semi-supervised remote sensing image change detection and prediction model is as follows:

[0131] 1) Obtain the remote sensing image change detection dataset and divide it into a labeled training set and an unlabeled training set. The label refers to the pixel-level annotation of the change area between two bi-temporal images, with the change area in white and the unchange area in black.

[0132] 2) Constructing a teacher network model With student network model ,

[0133] 3) Training the teacher network model ,

[0134] Labeled training set The network model is trained by inputting the teacher network model, and the network parameters are updated through backpropagation. The optimal parameters are saved as the teacher model.

[0135] 4) Train the student network model ,

[0136] The training set will be processed by the labeled training set. The parameters of the trained teacher model are used as the student network model. The initialization parameters will be used to initialize the unlabeled training set. Simultaneously input the student network model The prediction results of the teacher model and the student network model are obtained respectively.

[0137] The predictions from the teacher model are then used as pseudo-labels and input into the student network model for further optimization. Finally, the parameters of the optimized student network model are synchronized to the teacher model using a moving exponential average.

[0138] The student network model was continued to be trained and the teacher model parameters were updated using the unlabeled training set until training was terminated. The optimal student network model parameters were saved as the remote sensing image change detection model.

[0139] 5) Input the test set into the remote sensing image change detection model, and verify the performance of the remote sensing image change detection model by the difference between the label data in the test set and the model prediction results.

[0140] The device also includes a detection output module, which is used to input the remote sensing image to be detected into the semi-supervised remote sensing image change detection and prediction model to obtain the detection result of the remote sensing image change.

[0141] The specific working processes of each module described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. The division of modules is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple modules may be combined or integrated into another system.

[0142] Based on the same inventive concept, the present invention provides a computer system including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is loaded onto the processor, it implements the steps of the above-described semi-supervised remote sensing image change detection method based on multi-scale feature fusion.

[0143] Based on the same inventive concept, the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described semi-supervised remote sensing image change detection method based on multi-scale feature fusion.

[0144] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0145] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0146] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0147] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0148] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A semi-supervised remote sensing image change detection method based on multi-scale feature fusion, characterized in that, The application relates to a method for constructing a semi-supervised remote sensing image change detection prediction model. The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps:

2. The semi-supervised remote sensing image change detection method based on multi-scale feature fusion according to claim 1, characterized in that, The method comprises the following steps: For input remote sensing image before change image and after change image , feature extraction is performed to obtain two feature maps and , Subtract semantic feature map output by feature extraction layer , .

3. The semi-supervised remote sensing image change detection method based on multi-scale feature fusion according to claim 2, characterized in that, The method comprises the following steps: , Global average pooling, 1x1 convolution, Relu function, 1x1 convolution and Sigmoid function are performed on the feature map ;​ ;​​​ ;​​​ Concatenate the , and to obtain the feature map ; will be described below. After global max pooling, 1x1 convolution and Relu operation, the enhanced feature map is obtained by adding , .

4. The semi-supervised remote sensing image change detection method based on multi-scale feature fusion according to claim 3, characterized in that, The method comprises the following steps: right After upsampling, 1×1 convolution, ReLU and global average pooling, and The feature maps are obtained by performing addition operations followed by global average pooling. , After upsampling, 1x1 convolution, Relu and global average pooling, the feature map is added to the feature map , and the feature map is obtained after global average pooling. right After upsampling, 1×1 convolution, ReLU and global average pooling, and The feature maps are obtained by performing addition operations followed by global average pooling. , right After upsampling, 1×1 convolution, ReLU and global average pooling, and The feature maps are obtained by performing addition operations followed by global average pooling. , Will Perform downsampling and 1×1 convolution, Perform downsampling and 1×1 convolution. Perform upsampling and 1×1 convolution. After upsampling and 1×1 convolution, and with The feature map is obtained by performing a stitching operation. , Will The final feature map of the change region is obtained after a 1×1 convolution. .

5. The semi-supervised remote sensing image change detection method based on multi-scale feature fusion according to claim 4, characterized in that, The prediction module is configured to output a prediction result of the change region feature map The prediction result of the remote sensing image change is output by upsampling to the size of the original image and using a 1x1 convolution.

6. The semi-supervised remote sensing image change detection method based on multi-scale feature fusion according to claim 5, characterized in that, The method comprises the following steps: The method comprises the following steps: Selecting a labeled training set , and respectively refer to the image before the remote sensing image change and the image after the remote sensing image change, is the label information; Will and The input is processed by the semantic feature extraction module in the teacher network model. and ,Will and Subtraction yields the feature map. Feature map Enhanced feature maps are obtained through the multi-scale feature enhancement module. The feature map of the changed region is obtained through the multi-scale feature fusion module. The prediction module outputs the detection results of image changes. ; The method comprises the following steps: The method comprises the following steps: ， wherein, N is the number of data in the labeled training set.

7. The semi-supervised remote sensing image change detection method based on multi-scale feature fusion according to claim 5, characterized in that, The method comprises the following steps: The method comprises the following steps: selecting a set of unlabeled training sets , Will And In the student network model, the semantic feature extraction module obtains And Subtract And To obtain the feature map , the feature map Through the multi-scale feature enhancement module, the enhanced feature map is obtained , the change area feature map is obtained through the multi-scale feature fusion module , the detection result of image change is output through the prediction module ; the group Data is simultaneously input into the trained teacher model to obtain the output result As a pseudo label; The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following wherein are teacher model parameters, are student network model parameters; Continue to train in the above manner using the unlabeled training set until a maximum number of iterations is reached, save the optimal student network model parameters, and take the student network model after training as the remote sensing image change detection model.

8. A semi-supervised remote sensing image change detection device based on multi-scale feature fusion, characterized in that, The device for implementing the semi-supervised remote sensing image change detection method based on multi-scale feature fusion according to any one of claims 1 to 7 comprises: a semi-supervised remote sensing image change detection prediction model training module for training a semi-supervised remote sensing image change detection prediction model; The semi-supervised remote sensing image change detection prediction model comprises a teacher network model and a student network model, and both the teacher network model and the student network model are composed of a semantic feature extraction module, a multi-scale feature enhancement module, a multi-scale feature fusion module, and a prediction module; the semantic feature extraction module is configured to extract a multi-scale semantic feature map of an input remote sensing image; the multi-scale feature enhancement module is configured to enhance multi-level feature information of the obtained semantic feature map; the multi-scale feature fusion module is configured to interactively fuse semantic features of remote sensing images of different scales to obtain a change region feature map; and the prediction module is configured to output a remote sensing image change detection result based on the change region feature map. The training method comprises the following steps: obtaining a remote sensing image change detection dataset and dividing the dataset into a labeled training set and an unlabeled training set; the label refers to pixel-level labeling of a change region between two double-time-phase images, with the change region being white and the unchanged region being black; constructing a teacher network model and a student network model; training the teacher network model using the labeled training set and initializing the student network model parameters with the trained teacher network model parameters; inputting the student network model and the trained teacher network model using the unlabeled training set to obtain respective prediction results of the teacher network model and the student network model; optimizing the student network model parameters with the prediction result of the teacher network model as a pseudo label and updating the teacher network model parameters; continuing to train the student network model and update the teacher model parameters using the unlabeled training set until the training is terminated, and saving the optimal student network model parameters as a remote sensing image change detection model; The device further comprises a detection output module configured to input a remote sensing image to be detected into the semi-supervised remote sensing image change detection prediction model to obtain a remote sensing image change detection result; the remote sensing image to be detected refers to double-time-phase remote sensing images generated by photographing the same region at different times, including a remote sensing image before a change and a remote sensing image after a change; and the detection result refers to a black-and-white picture of a change region with a size equal to that of the input remote sensing image, wherein black represents a background region without a change and white represents a region with a change.

9. A computer-readable storage medium storing one or more programs, the one or more programs comprising instructions that when executed by a computer cause the computer to perform a method of any of claims 1-8. The one or more programs include instructions that, when executed by a computing device, cause the computing device to perform any of the semi-supervised remote sensing image change detection methods based on multi-scale feature fusion according to claims 1 to 7.

10. A computing device, comprising: The computer program product comprises one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs comprise instructions for executing any one of the methods for multi-scale feature fusion based semi-supervised remote sensing image change detection according to claims 1 to 7.

Images