Semi-supervised remote sensing image change detection method and device based on multi-scale feature fusion

Abstract

The invention provides a semi-supervised remote sensing image change detection method and device based on multi-scale feature fusion, and the method comprises the steps: obtaining a labeled training set and a non-labeled training set of remote sensing image change detection data, constructing a teacher network model and a student network model, training the teacher network model through the labeled training set, and obtaining a student network model; initializing student network model parameters according to the trained teacher network model parameters, and inputting the student network model and the teacher network model by adopting a non-label training set to obtain respective prediction results; using a prediction result of the teacher network model as a pseudo label to optimize parameters of the student network model, updating the parameters of the teacher network model, and finally obtaining a remote sensing image change detection model through iterative training; and performing remote sensing image change detection based on the detection model. According to the method, spatial information of different scales can be extracted, feature enhancement is carried out, feature information of different scales is effectively fused, and the prediction performance of the model is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of remote sensing image processing, and in particular relates to a semi-supervised remote sensing image change detection method and device based on multi-scale feature fusion. Background Art

[0002] In recent years, with the development of remote sensing satellite technology, the amount of data obtained from remote sensing images has become increasingly abundant, and change detection based on remote sensing images has become one of the research hotspots. Change detection refers to the process of identifying and analyzing the changes in the surface or objects in time and space in images or data obtained at different times or under different conditions using remote sensing technology or other related technologies.

[0003] In traditional remote sensing image change detection, the acquisition of high-quality annotated data is costly and time-consuming, especially in large-scale, long-time series monitoring, where fully supervised methods are difficult to meet the needs. Remote sensing images are huge in size, and a single image contains a large number of targets. Manual annotation takes a lot of time, and due to the limitations of factors such as the annotator's experience and skill level, the annotation results may differ from the actual change area, making it difficult to ensure the accuracy of the annotated samples, which will affect the accuracy and reliability of subsequent change detection. Summary of the invention

[0004] In view of the problems of low utilization rate of unlabeled data sample information and misleading pseudo-labels in the field of semi-supervised change detection, the purpose of the present invention is to provide a semi-supervised remote sensing image change detection method and device based on multi-scale feature fusion. By adopting a student-teacher network structure, the teacher model generates pseudo-labels to assist the student model in training, and adopts a feature enhancement module and a multi-scale feature fusion module to capture effective information in unlabeled data, so as to achieve the purpose of accurately extracting changed areas during change detection.

[0005] To achieve the above object, the technical solution adopted by the present invention is as follows:

[0006] In a first aspect, the present invention provides a semi-supervised remote sensing image change detection method based on multi-scale feature fusion, comprising:

[0007] Pre-build a semi-supervised remote sensing image change detection prediction model;

[0008] The remote sensing image to be detected is input into the above-mentioned semi-supervised remote sensing image change detection prediction model to obtain the detection result of the remote sensing image change; the remote sensing image to be detected refers to a dual-phase remote sensing image generated by photographing the same area at different times, including a remote sensing image before the change and a remote sensing image after the change; the detection result refers to a black-and-white image of the change area with a size equal to that of the input remote sensing image, wherein black represents the background area that has not changed, and white represents the area that has changed;

[0009] The pre-built semi-supervised remote sensing image change detection prediction model includes:

[0010] Obtain a remote sensing image change detection dataset and divide it into a labeled training set and an unlabeled training set. The label refers to pixel-level annotation of the changed area between two dual-phase images, where the changed area is white and the unchanged area is black.

[0011] Constructing a teacher network model and a student network model, wherein the teacher network model and the student network model are both composed of a semantic feature extraction module, a multi-scale feature enhancement module, a multi-scale feature fusion module and a prediction module; the teacher network model and the student network model are used to extract a multi-scale semantic feature map of an input remote sensing image and perform feature enhancement and interactive fusion to obtain a prediction result of a remote sensing image change;

[0012] The teacher network model is trained using a labeled training set, and the student network model parameters are initialized using the trained teacher network model parameters, and the student network model and the trained teacher network model are input into an unlabeled training set to obtain prediction results of the teacher network model and the student network model respectively;

[0013] The prediction results of the teacher network model are used as pseudo labels to optimize the student network model parameters and update the teacher network model parameters. The unlabeled training set is continued to be used to train the student network model and update the teacher model parameters until the training is terminated. The optimal student network model parameters are saved as the remote sensing image change detection model.

[0014] Preferably, the semantic feature extraction module adopts a deep learning network, and the semantic feature extraction module includes 5 feature extraction layers.

[0015] For the image before the input remote sensing image changes And the image after the change , perform feature extraction to obtain two feature maps and ,use minus Semantic feature map as output of feature extraction layer , .

[0016] Preferably, the multi-scale feature enhancement module is used to enhance the multi-level feature information of the semantic feature map, and the data processing is as follows:

[0017] right , Perform global average pooling, 1×1 convolution, Relu function, 1×1 convolution and Sigmoid function to obtain the feature map ;

[0018] right , Perform global average pooling, 3×3 convolution, Relu function, 1×1 convolution and Sigmoid function to obtain the feature map ;

[0019] right , Perform global average pooling, 5×5 convolution, Relu function, 1×1 convolution and Sigmoid function to obtain the feature map ;

[0020] Will , and Perform concatenation to obtain feature maps ;

[0021] Will After global maximum pooling, 1×1 convolution and ReLU operations, Add to get the enhanced feature map , .

[0022] Preferably, the multi-scale feature fusion module is used to interact the semantic features of remote sensing images of different scales to obtain a feature map of the changed area, and the data processing is as follows:

[0023] right After upsampling, 1×1 convolution, ReLU and global average pooling, Perform the addition operation and obtain the feature map through global average pooling ,

[0024] right After upsampling, 1×1 convolution, ReLU and global average pooling, Perform the addition operation and obtain the feature map through global average pooling ,

[0025] right After upsampling, 1×1 convolution, ReLU and global average pooling, Perform the addition operation and obtain the feature map through global average pooling ,

[0026] right After upsampling, 1×1 convolution, ReLU and global average pooling, Perform the addition operation and obtain the feature map through global average pooling ,

[0027] Will Perform downsampling and 1×1 convolution, Perform downsampling and 1×1 convolution, Perform upsampling and 1×1 convolution, After upsampling and 1×1 convolution, Perform concatenation to obtain feature maps ,

[0028] Will After a 1×1 convolution, the final feature map of the changed area is obtained .

[0029] Preferably, the prediction module is used to convert the change region feature map The image is restored to its original size by upsampling, and a 1×1 convolution is used to output the prediction results of the remote sensing image changes.

[0030] Preferably, the training process of the teacher network model is:

[0031] Initialize the teacher network model parameters;

[0032] Select a labeled training set , and They refer to the image before and after the remote sensing image changes. It is the label information;

[0033] Will and Input into the teacher network model, and obtain through the semantic feature extraction module and ,Will and Subtract and get the feature map , feature map The enhanced feature map is obtained by the multi-scale feature enhancement module , the feature map of the changing area is obtained through the multi-scale feature fusion module , the prediction module outputs the detection result of image changes ;

[0034] Calculate the loss according to the loss function and update the network model parameters through back propagation; through continuous iterative training until the maximum number of iterations is reached, save the optimal network model parameters as the teacher model;

[0035] The loss function is:

[0036] ,

[0037] in, is the number of labeled data in the training set.

[0038] Preferably, the training process of the student network model is:

[0039] Use the parameters of the trained teacher model to initialize the student network model parameters;

[0040] Select an unlabeled training set ,

[0041] Will and Input into the student network model, and obtain through the semantic feature extraction module and ,Will and Subtract and get the feature map , feature map The enhanced feature map is obtained by the multi-scale feature enhancement module , the feature map of the changing area is obtained through the multi-scale feature fusion module , the prediction module outputs the detection result of image changes ; This group The data is simultaneously input into the trained teacher model to obtain the output result As pseudo-labels;

[0042] Calculate the loss based on the pseudo-labels and update the student network model parameters through back propagation;

[0043] The teacher model parameters are updated by sliding exponential averaging. The update method is:

[0044] ,in are the teacher model parameters, are the parameters of the student network model;

[0045] Continue to use the unlabeled training set to iterate the training in the above manner until the maximum number of iterations is reached, save the optimal student network model parameters, and use the trained student network model as the remote sensing image change detection model.

[0046] In a second aspect, the present invention provides a semi-supervised remote sensing image change detection device based on multi-scale feature fusion, which is used to implement the above-mentioned semi-supervised remote sensing image change detection method based on multi-scale feature fusion, and the device includes:

[0047] Semi-supervised remote sensing image change detection prediction model training module, used to train the semi-supervised remote sensing image change detection prediction model;

[0048] The semi-supervised remote sensing image change detection prediction model includes a teacher network model and a student network model, and the teacher network model and the student network model are both composed of a semantic feature extraction module, a multi-scale feature enhancement module, a multi-scale feature fusion module and a prediction module; the semantic feature extraction module is used to extract a multi-scale semantic feature map of an input remote sensing image; the multi-scale feature enhancement module is used to enhance the multi-level feature information of the acquired semantic feature map; the multi-scale feature fusion module is used to interactively fuse the semantic features of remote sensing images of different scales to obtain a change area feature map; the prediction module is used to output the detection result of the remote sensing image change based on the change area feature map;

[0049] The training method is as follows: obtaining a remote sensing image change detection data set, and dividing it into a labeled training set and an unlabeled training set, wherein the label refers to pixel-level annotation of the changed area between two dual-phase images, where the changed area is white and the unchanged area is black; constructing a teacher network model and a student network model; using the labeled training set to train the teacher network model, and using the trained teacher network model parameters to initialize the student network model parameters, using the unlabeled training set to input the student network model and the trained teacher network model, and obtaining the prediction results of the teacher network model and the student network model respectively; using the prediction results of the teacher network model as pseudo-labels to optimize the student network model parameters, and updating the teacher network model parameters, continuing to use the unlabeled training set to train the student network model and update the teacher model parameters until the training is terminated, and saving the optimal student network model parameters as the remote sensing image change detection model;

[0050] It also includes: a detection output module, which is used to input the remote sensing image to be detected into the semi-supervised remote sensing image change detection prediction model to obtain the detection result of the remote sensing image change; the remote sensing image to be detected refers to a dual-phase remote sensing image generated by shooting the same area at different times, including a remote sensing image before the change and a remote sensing image after the change; the detection result refers to a black-and-white picture of the change area with a size equal to the size of the input remote sensing image, wherein black represents the background area that has not changed, and white represents the area that has changed.

[0051] In a third aspect, the present invention provides a computer-readable storage medium storing one or more programs, wherein the one or more programs include instructions, which, when executed by a computing device, enable the computing device to perform any of the above-mentioned semi-supervised remote sensing image change detection methods based on multi-scale feature fusion.

[0052] In a fourth aspect, the present invention provides a computing device comprising one or more processors, a memory and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for executing any of the above-mentioned semi-supervised remote sensing image change detection methods based on multi-scale feature fusion.

[0053] Compared with the prior art, the present invention has the following beneficial effects:

[0054] The present invention provides a semi-supervised remote sensing image change detection method based on multi-scale feature fusion, which can be applied to change detection tasks in scenarios with limited labeled data. The present invention extracts spatial information of different scales, further enhances the extracted feature map, obtains multi-scale feature information using different receptive fields, and effectively fuses the feature information of different scales in a multi-scale feature fusion module, thereby further accurately capturing the boundaries of the changed areas of the remote sensing image, generating accurate annotation results of the areas where real changes occur, and improving the prediction performance of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 Schematic diagram of the training process of the semi-supervised remote sensing image change detection model in an embodiment of the present invention;

[0056] Figure 2 is a structural diagram of a semi-supervised remote sensing image change detection model provided by an embodiment of the present invention;

[0057] Figure 3 is a schematic diagram of the data processing process of the feature enhancement module in an embodiment of the present invention;

[0058] Figure 4 It is a schematic diagram of the data processing process of the multi-scale feature fusion module in an embodiment of the present invention. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments and the accompanying drawings. Here, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0060] It should also be noted that, in order to avoid obscuring the present invention due to unnecessary details, only structures and / or processing steps closely related to the solutions according to the present invention are shown in the accompanying drawings, while other details that are not closely related to the present invention are omitted.

[0061] It should be emphasized that the term “include / comprises” when used herein refers to the presence of features, elements, steps or components, but does not exclude the presence or addition of one or more other features, elements, steps or components.

[0062] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the accompanying drawings, the same reference numerals represent the same or similar components, or the same or similar steps.

[0063] It should be emphasized here that the step marks mentioned below are not intended to limit the order of the steps, but should be understood that the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.

[0064] A first embodiment of the present invention provides a semi-supervised remote sensing image change detection method based on multi-scale feature fusion, comprising:

[0065] Pre-build a semi-supervised remote sensing image change detection prediction model;

[0066] The remote sensing image to be detected is input into the above-mentioned semi-supervised remote sensing image change detection prediction model to obtain the detection result of the remote sensing image change, wherein the remote sensing image to be detected is a dual-phase remote sensing image generated by a drone or a satellite shooting the same area at different times, wherein the image acquired for the first time is the remote sensing image before the change, and the image acquired for the second time is the remote sensing image after the change; the detection result is a black-and-white picture of the change area whose size is equal to the size of the input remote sensing image, wherein black represents the background area that has not changed, and white represents the area that has changed between the two input remote sensing images.

[0067] Among them, a semi-supervised remote sensing image change detection prediction model is pre-built, see Figure 1 The specific implementation process is as follows:

[0068] 1) Obtain a remote sensing image change detection dataset and divide it into labeled training sets and unlabeled training sets. Labels refer to pixel-level annotation of the changed areas between two dual-phase images, with the changed areas being white and the unchanged areas being black.

[0069] 2) Constructing the teacher network model Network model with students ,

[0070] See also Figure 2, teacher network model Network model with students They are composed of semantic feature extraction module, multi-scale feature enhancement module, multi-scale feature fusion module and prediction module; teacher network model Network model with students It is used to extract the semantic feature map of the input remote sensing image, enhance the multi-level feature information of the acquired semantic feature map, interactively fuse the semantic features of remote sensing images in different scale change areas, obtain the feature map of the change area and output the final prediction result;

[0071] 3) Training the teacher network model ,

[0072] The labeled training set Input the teacher network model for training, update the network parameters through back propagation, and save the optimal parameters as the teacher model;

[0073] 4) Train the student network model ,

[0074] Will be through the labeled training set The parameters of the trained teacher model are used as the student network model The initialization parameters of the unlabeled training set Input the student network model at the same time and the trained teacher model to obtain the prediction results of the teacher model and the student network model respectively;

[0075] The prediction results of the teacher model are then input into the student network model as pseudo labels, and the student network model is further optimized and trained. In each round of iteration, the parameters of the optimized student network model are synchronized to the teacher model through sliding exponential averaging. The unlabeled training set is continued to be used to train the student network model and update the teacher model parameters until the training is terminated. The optimal parameters of the student network model are saved, and the trained student network model is used as a remote sensing image change detection model.

[0076] 5) Input the test set into the remote sensing image change detection model, and test the performance of the remote sensing image change detection model by the difference between the labeled data in the test set and the model prediction results.

[0077] In this embodiment, the Levir-CD dataset is used as the remote sensing image change detection dataset.

[0078] It should be noted that during the training process, the network parameters are updated through back propagation to make the loss function reach the smallest possible value. The maximum number of iterations is set. When the maximum number of iterations is reached, the model parameters at this time are saved as the optimal network parameters and imported into the network model as a remote sensing image change detection prediction model.

[0079] It should be noted that, in one embodiment, the maximum number of iterations of the teacher network model is 100 times, and the maximum number of iterations of the student network model is 50 times.

[0080] It should be noted that the sliding exponential average algorithm used is used to transfer the parameters of the student network model to the teacher network model, including:

[0081] The parameters of the teacher network model are , the parameters of the student network model are ,but,

[0082] ,

[0083] In this embodiment, The value is 0.95.

[0084] In one embodiment, a remote sensing image change detection dataset is obtained and divided into a labeled training set and an unlabeled training set, as follows:

[0085] 11) Obtaining remote sensing image change detection dataset ,Will Divide into training sets according to the ratio of 5:5 and test set ;

[0086] 12) The training set Select 10% of the data sample As a labeled training set, the remaining training set is used as an unlabeled training set ,in , and They refer to the image before and after the remote sensing image changes. Label information It is to perform pixel-level annotation of the changed area between two dual-phase images. The changed area is white and the unchanged area is black.

[0087] In one embodiment, the semantic feature extraction module extracts the semantic features of the input image based on different receptive fields based on convolution operation, batch normalization, ReLU function and maximum pooling operation. The specific process is as follows:

[0088] 21) Assume that the input remote sensing image before the change And the image after the change Input size is ,in In this embodiment, bit 224, Also 224, which is the height and width of the image;

[0089] 22) Input the preprocessed remote sensing image into the semantic feature extraction module. The semantic feature extraction module in this embodiment is a ResNet-50 network, and can also be other types of feature extraction networks such as VGG16, DeeplabV3 and other deep learning networks to obtain two feature maps of each feature extraction layer. and ,use minus As the output feature of the feature extraction layer ;

[0090] 23) The network structure of the semantic feature extraction module is divided into 5 layers, and the feature map that can be extracted in each layer is ;in , , Respectively The feature extraction layer outputs the height, width, and number of channels of the feature map.

[0091] In one embodiment, the multi-scale feature enhancement module is used to enhance the multi-level feature information of the semantic feature map, which helps to accurately extract the fuzzy boundary and improve the model performance. Figure 3 ,The data processing process of the multi-scale feature enhancement module is as follows:

[0092] For each layer’s feature map Perform convolution operations with convolution kernels of different sizes, including:

[0093] 31) Yes Perform global average pooling, 1×1 convolution, Relu function, 1×1 convolution and Sigmoid function to obtain the feature map , expressed as ;

[0094] 32) Yes Perform global average pooling, 3×3 convolution, Relu function, 1×1 convolution and Sigmoid function to obtain the feature map , expressed as ;

[0095] 33) Yes Perform global average pooling, 5×5 convolution, Relu function, 1×1 convolution and Sigmoid function to obtain the feature map , expressed as ;in represents global average pooling, represents a 5×5 convolution, represents the Relu function, represents 1×1 convolution, Represents the Sigmoid function;

[0096] 34) , and Perform concatenation to obtain feature maps , expressed as: ,in Represents a splicing operation,

[0097] Then After global maximum pooling, 1×1 convolution and ReLU operations, Add together to get the enhanced feature map of this layer , expressed as:

[0098] ,

[0099] in represents the global maximum pooling, represents the Relu function, represents 1×1 convolution, Indicates addition.

[0100] In one embodiment, the multi-scale feature fusion module is used to interact the semantic features of remote sensing images of regions with different scales to obtain a feature map of the region with different scales. Figure 4 , The data processing process of the multi-scale feature fusion module is as follows:

[0101] 41) Yes After upsampling, 1×1 convolution, ReLU and global average pooling, Perform the addition operation and obtain the feature map through global average pooling , expressed as:

[0102] ;

[0103] 42) Yes After upsampling, 1×1 convolution, ReLU and global average pooling, Perform the addition operation and obtain the feature map through global average pooling , expressed as:

[0104] ;

[0105] 43) Yes After upsampling, 1×1 convolution, ReLU and global average pooling, Perform the addition operation and obtain the feature map through global average pooling , expressed as:

[0106] ;

[0107] 44) Yes After upsampling, 1×1 convolution, ReLU and global average pooling, Perform the addition operation and obtain the feature map through global average pooling , expressed as:

[0108] ,

[0109] in Represents an upsampling operation;

[0110] 45) Perform downsampling and 1×1 convolution, Perform downsampling and 1×1 convolution, Perform upsampling and 1×1 convolution, After upsampling and 1×1 convolution, Perform concatenation to obtain feature maps , expressed as:

[0111] ,

[0112] in represents the downsampling operation,

[0113] Will After a 1×1 convolution, the final feature map is obtained , expressed as; .

[0114] In one embodiment, the prediction module converts The feature map is restored to the size of the original image, and a 1×1 convolution is used to output the final prediction result, which is a black and white picture of the changed area with the same size as the input remote sensing image, where black represents the background area that has not changed, and white represents the area that has changed between the two input remote sensing images.

[0115] In one embodiment, the training process of the teacher network model is:

[0116] 51) Initialize the teacher network model parameters;

[0117] 52) Select a labeled training set ,

[0118] 53) and Input into the teacher network model, and obtain through the semantic feature extraction module and ,Will and Subtract and get the feature map , feature map The enhanced feature map is obtained by the multi-scale feature enhancement module , the feature map of the changing area is obtained through the multi-scale feature fusion module ;Will The prediction module obtains the detection result of the input image change ;

[0119] 54) According to the loss function Calculate the loss and update the parameters through back-propagation;

[0120] 55) Iterate the training in the above way until the maximum number of iterations is reached, and save the optimal parameters as the teacher model.

[0121] In one embodiment, the training process of the student network model is:

[0122] 61) Use the parameters of the teacher model to initialize the parameters of the student network model;

[0123] 62) Select an unlabeled training set ,

[0124] 63) and Input into the student network model, and obtain through the semantic feature extraction module and ,Will and Subtract and get the feature map , feature map The enhanced feature map is obtained by the multi-scale feature enhancement module , the feature map is obtained through the multi-scale feature fusion module ;Will The change detection result of the input is obtained through the prediction module ; This group The data is simultaneously input into the teacher model to obtain the output result As pseudo-labels;

[0125] 64) According to the loss function Calculate the loss and update the student network model parameters through back propagation;

[0126] 65) Sliding Exponential Average Update the parameters of the teacher model, where the teacher model parameters are , the parameters of the student network model are ;

[0127] 66) Repeat steps 62)-65) for iterative training until the maximum number of iterations is reached, and save the optimal student network model parameters as the remote sensing image change detection model.

[0128] Based on the above inventive concept, the present invention also provides a semi-supervised remote sensing image change detection device based on multi-scale feature fusion, which is used to implement the above-mentioned semi-supervised remote sensing image change detection method based on multi-scale feature fusion, and the device includes:

[0129] Semi-supervised remote sensing image change detection prediction model training module, used to train the semi-supervised remote sensing image change detection prediction model, the semi-supervised remote sensing image change detection prediction model includes a teacher network model and a student network model, see Figure 2 ,The teacher network model and the student network model are composed of a semantic feature extraction module, a multi-scale feature enhancement module, a multi-scale feature fusion module and a prediction module; the semantic feature extraction module is used to extract the multi-scale semantic feature map of the input remote sensing image; the multi-scale feature enhancement module is used to enhance the multi-level feature information of the acquired semantic feature map; the multi-scale feature fusion module is used to interactively fuse the semantic features of remote sensing images in different scale change areas to obtain the feature map of the change area; the prediction module is used to output the final prediction result;

[0130] Among them, the training process of the semi-supervised remote sensing image change detection prediction model is as follows:

[0131] 1) Obtain a remote sensing image change detection dataset and divide it into labeled training sets and unlabeled training sets. Labels refer to pixel-level annotation of the changed areas between two dual-phase images, with the changed areas being white and the unchanged areas being black.

[0132] 2) Constructing the teacher network model Network model with students ,

[0133] 3) Training the teacher network model ,

[0134] The labeled training set Input the teacher network model for training, update the network parameters through back propagation, and save the optimal parameters as the teacher model;

[0135] 4) Train the student network model ,

[0136] Will be through the labeled training set The parameters of the trained teacher model are used as the student network model The initialization parameters of the unlabeled training set Input the student network model at the same time and the teacher model, and obtain the prediction results of the teacher model and the student network model respectively;

[0137] The prediction results of the teacher model are then input into the student network model as pseudo labels, the student network model is further optimized, and the parameters of the optimized student network model are synchronized to the teacher model through sliding exponential averaging.

[0138] Continue to use the unlabeled training set to train the student network model and update the teacher model parameters until the training is terminated, and save the optimal student network model parameters as the remote sensing image change detection model.

[0139] 5) Input the test set into the remote sensing image change detection model, and test the performance of the remote sensing image change detection model by the difference between the labeled data in the test set and the model prediction results.

[0140] The device also includes a detection output module, which is used to input the remote sensing image to be detected into the above-mentioned semi-supervised remote sensing image change detection prediction model to obtain the detection result of the remote sensing image change.

[0141] The specific working process of each module described above can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here. The module division is only a logical function division, and there may be other division methods in actual implementation, such as multiple modules can be combined or integrated into another system.

[0142] Based on the same inventive concept, the present invention provides a computer system, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is loaded into the processor, the steps of the above-mentioned semi-supervised remote sensing image change detection method based on multi-scale feature fusion are implemented.

[0143] Based on the same inventive concept, the present invention provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the steps of the above-mentioned semi-supervised remote sensing image change detection method based on multi-scale feature fusion.

[0144] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0145] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0146] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0147] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0148] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A semi-supervised remote sensing image change detection method based on multi-scale feature fusion, characterized in that: include: Pre-build a semi-supervised remote sensing image change detection prediction model; The remote sensing image to be detected is input into the above-mentioned semi-supervised remote sensing image change detection prediction model to obtain the detection result of the remote sensing image change; the remote sensing image to be detected refers to a dual-phase remote sensing image generated by photographing the same area at different times, including a remote sensing image before the change and a remote sensing image after the change; the detection result refers to a black-and-white image of the change area with a size equal to that of the input remote sensing image, wherein black represents the background area that has not changed, and white represents the area that has changed; The pre-built semi-supervised remote sensing image change detection prediction model includes: Obtain a remote sensing image change detection dataset and divide it into a labeled training set and an unlabeled training set. The label refers to pixel-level annotation of the changed area between two dual-phase images, where the changed area is white and the unchanged area is black. Constructing a teacher network model and a student network model, wherein the teacher network model and the student network model are both composed of a semantic feature extraction module, a multi-scale feature enhancement module, a multi-scale feature fusion module and a prediction module; the teacher network model and the student network model are used to extract a multi-scale semantic feature map of an input remote sensing image and perform feature enhancement and interactive fusion to obtain a prediction result of a remote sensing image change; The teacher network model is trained using a labeled training set, and the student network model parameters are initialized using the trained teacher network model parameters, and the student network model and the trained teacher network model are input into an unlabeled training set to obtain prediction results of the teacher network model and the student network model respectively; The prediction results of the teacher network model are used as pseudo labels to optimize the student network model parameters and update the teacher network model parameters. The unlabeled training set is continued to be used to train the student network model and update the teacher model parameters until the training is terminated. The optimal student network model parameters are saved as the remote sensing image change detection model.

2. According to claim 1, a semi-supervised remote sensing image change detection method based on multi-scale feature fusion is characterized in that: The semantic feature extraction module adopts a deep learning network, and the semantic feature extraction module includes 5 layers of feature extraction layers. For the image before the input remote sensing image changes And the image after the change , perform feature extraction to obtain two feature maps and ,use minus Semantic feature map as output of feature extraction layer , .

3. The method for semi-supervised remote sensing image change detection based on multi-scale feature fusion according to claim 2, characterized in that: The multi-scale feature enhancement module is used to enhance the multi-level feature information of the semantic feature map, and the data processing is as follows: right , Perform global average pooling, 1×1 convolution, Relu function, 1×1 convolution and Sigmoid function to obtain the feature map ; right , Perform global average pooling, 3×3 convolution, Relu function, 1×1 convolution and Sigmoid function to obtain the feature map ; right , Perform global average pooling, 5×5 convolution, Relu function, 1×1 convolution and Sigmoid function to obtain the feature map ; Will , and Perform concatenation operation to obtain feature map ; Will After global maximum pooling, 1×1 convolution and ReLU operations, Add to get the enhanced feature map , .

4. The method for semi-supervised remote sensing image change detection based on multi-scale feature fusion according to claim 3 is characterized in that: The multi-scale feature fusion module is used to interact the semantic features of remote sensing images of different scales to obtain a feature map of the changed area. The data processing is as follows: right After upsampling, 1×1 convolution, ReLU and global average pooling, Perform the addition operation and obtain the feature map through global average pooling , right After upsampling, 1×1 convolution, ReLU and global average pooling, Perform the addition operation and obtain the feature map through global average pooling , right After upsampling, 1×1 convolution, ReLU and global average pooling, Perform the addition operation and obtain the feature map through global average pooling , right After upsampling, 1×1 convolution, ReLU and global average pooling, Perform the addition operation and obtain the feature map through global average pooling , Will Perform downsampling and 1×1 convolution, Perform downsampling and 1×1 convolution, Perform upsampling and 1×1 convolution, After upsampling and 1×1 convolution, Perform concatenation operation to obtain feature map , Will After a 1×1 convolution, the final feature map of the changed area is obtained .

5. The method for semi-supervised remote sensing image change detection based on multi-scale feature fusion according to claim 4, characterized in that: The prediction module is used to transform the change region feature map The image is restored to its original size by upsampling, and a 1×1 convolution is used to output the prediction results of the remote sensing image changes.

6. The method for semi-supervised remote sensing image change detection based on multi-scale feature fusion according to claim 5, characterized in that: The training process of the teacher network model is: Initialize the teacher network model parameters; Select a labeled training set , and They refer to the image before and after the remote sensing image changes. It is the label information; Will and Input into the teacher network model, and obtain through the semantic feature extraction module and ,Will and Subtract and get the feature map , feature map The enhanced feature map is obtained by the multi-scale feature enhancement module , the feature map of the changing area is obtained through the multi-scale feature fusion module , the prediction module outputs the detection result of image changes ; Calculate the loss according to the loss function and update the network model parameters through back propagation; through continuous iterative training until the maximum number of iterations is reached, save the optimal network model parameters as the teacher model; The loss function is: ， in, is the number of labeled data in the training set.

7. The method for semi-supervised remote sensing image change detection based on multi-scale feature fusion according to claim 5, characterized in that: The training process of the student network model is: Use the parameters of the trained teacher model to initialize the student network model parameters; Select an unlabeled training set , Will and Input into the student network model, and obtain through the semantic feature extraction module and ,Will and Subtract and get the feature map , feature map The enhanced feature map is obtained by the multi-scale feature enhancement module , the feature map of the changing area is obtained through the multi-scale feature fusion module , the prediction module outputs the detection result of image changes ; This group The data is simultaneously input into the trained teacher model to obtain the output result As pseudo-labels; Calculate the loss based on the pseudo-labels and update the student network model parameters through back propagation; The teacher model parameters are updated by sliding exponential averaging. The update method is: ,in are the teacher model parameters, are the parameters of the student network model; Continue to use the unlabeled training set to iterate the training in the above manner until the maximum number of iterations is reached, save the optimal student network model parameters, and use the trained student network model as the remote sensing image change detection model.

8. A semi-supervised remote sensing image change detection device based on multi-scale feature fusion, characterized in that: The device is used to implement the semi-supervised remote sensing image change detection method based on multi-scale feature fusion as described in any one of claims 1 to 7, comprising: Semi-supervised remote sensing image change detection prediction model training module, used to train the semi-supervised remote sensing image change detection prediction model; The semi-supervised remote sensing image change detection prediction model includes a teacher network model and a student network model, and the teacher network model and the student network model are both composed of a semantic feature extraction module, a multi-scale feature enhancement module, a multi-scale feature fusion module and a prediction module; the semantic feature extraction module is used to extract a multi-scale semantic feature map of an input remote sensing image; the multi-scale feature enhancement module is used to enhance the multi-level feature information of the acquired semantic feature map; the multi-scale feature fusion module is used to interactively fuse the semantic features of remote sensing images of different scales to obtain a change area feature map; the prediction module is used to output the detection result of the remote sensing image change based on the change area feature map; The training method is as follows: obtaining a remote sensing image change detection data set, and dividing it into a labeled training set and an unlabeled training set, wherein the label refers to pixel-level annotation of the changed area between two dual-phase images, where the changed area is white and the unchanged area is black; constructing a teacher network model and a student network model; using the labeled training set to train the teacher network model, and using the trained teacher network model parameters to initialize the student network model parameters, using the unlabeled training set to input the student network model and the trained teacher network model, and obtaining the prediction results of the teacher network model and the student network model respectively; using the prediction results of the teacher network model as pseudo-labels to optimize the student network model parameters, and updating the teacher network model parameters, continuing to use the unlabeled training set to train the student network model and update the teacher model parameters until the training is terminated, and saving the optimal student network model parameters as the remote sensing image change detection model; It also includes: a detection output module, which is used to input the remote sensing image to be detected into the semi-supervised remote sensing image change detection prediction model to obtain the detection result of the remote sensing image change; the remote sensing image to be detected refers to a dual-phase remote sensing image generated by shooting the same area at different times, including a remote sensing image before the change and a remote sensing image after the change; the detection result refers to a black-and-white picture of the change area with a size equal to the size of the input remote sensing image, wherein black represents the background area that has not changed, and white represents the area that has changed.

9. A computer-readable storage medium storing one or more programs, characterized in that: The one or more programs include instructions, which, when executed by a computing device, cause the computing device to perform any one of the semi-supervised remote sensing image change detection methods based on multi-scale feature fusion according to claims 1 to 7.

10. A computing device, characterized in that It includes one or more processors, a memory and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include instructions for executing any one of the semi-supervised remote sensing image change detection methods based on multi-scale feature fusion according to claims 1 to 7.

Images