Methods, devices, equipment, and storage media for detecting flood-damaged farmland.

Abstract

This invention provides a method, apparatus, device, and storage medium for detecting flood-damaged farmland, relating to the field of image processing technology. The method includes acquiring a first and a second remote sensing image of the farmland area to be detected; applying the first and second remote sensing images to a flood damage change detection model to obtain a flood damage map, wherein a Siamese deep convolutional neural network is constructed; a pre-trained flood damage change detection model is obtained by training the Siamese deep convolutional neural network based on labeled sample image data; a flood damage pseudo-label probability map is obtained by predicting unlabeled sample image data based on the pre-trained flood damage change detection model; and a semi-supervised training is performed on the pre-trained flood damage change detection model based on labeled sample image data, unlabeled sample image data, and the flood damage pseudo-label probability map to obtain a trained flood damage change detection model. This improves the accuracy of detecting changes in flood-damaged farmland.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method, apparatus, equipment, and storage medium for detecting flood-damaged farmland. Background Technology

[0002] In the field of remote sensing datasets, there are relatively few scenarios of varying degrees of damage to crops in farmland caused by floods. The types of assets damaged by floods are complex, and there is a large amount of unlabeled data. Due to the diverse and complex scenarios of farmland damaged after disasters, automatic change detection is a challenging task.

[0003] Currently, there is a lack of detailed monitoring studies on farmland with different degrees of damage after disasters. The remote sensing characteristics of farmland damaged by floods are diverse and complex. Compared to ordinary farmland or water bodies, damaged farmland, while still in a waterlogged state, has higher sediment and debris content, and the interaction between floodwaters and crops complicates remote sensing characteristics. Due to the impact of floods or continuous infiltration, some crops suffer varying degrees of damage. Examples include patchy and blocky farmland destroyed by floods and patchy and blocky farmland damaged by waterlogging. Completely damaged farmland exhibits either dry or wet soil conditions, and its remote sensing characteristics differ significantly due to varying moisture content. When crop straw remains on the farmland, it results in more complex spectral, shape, and texture features than completely damaged farmland. The fractal geometry of healthy farmland versus damaged farmland further increases the complexity of the segmentation task. Summary of the Invention

[0004] This invention provides a method, apparatus, equipment, and storage medium for detecting flood-damaged farmland, thereby addressing the deficiencies in existing methods for detecting flood-damaged farmland.

[0005] This invention provides a method for detecting flood-damaged farmland, the method comprising:

[0006] Acquire the first remote sensing image of the farmland area to be monitored before flood damage and the second remote sensing image after flood damage;

[0007] Based on the first and second remote sensing images of the farmland area to be detected, the flood damage change detection model is applied to obtain the flood damage map of the farmland area to be detected.

[0008] The flood damage change detection model was trained in the following way:

[0009] Constructing a twin deep convolutional neural network;

[0010] The twin deep convolutional neural network is trained based on labeled sample image data to obtain a pre-trained flood damage change detection model. The labeled sample image data includes at least one set of labeled farmland sample image pairs. The labeled farmland sample image pairs include a first farmland sample remote sensing image before flood damage, a second farmland sample remote sensing image after flood damage, and a flood damage real label map corresponding to the second farmland sample remote sensing image.

[0011] Based on the pre-trained flood damage change detection model, the unlabeled sample image data is predicted to obtain a flood damage pseudo-label probability map. The unlabeled sample image data includes at least one set of unlabeled farmland sample image pairs, which include a remote sensing image of the third farmland sample before flood damage and a remote sensing image of the fourth farmland sample after flood damage.

[0012] The pre-trained flood damage change detection model is semi-supervised based on the labeled sample image data, the unlabeled sample image data, and the flood damage pseudo-label probability map to obtain the trained flood damage change detection model.

[0013] According to the present invention, a method for detecting flood-damaged farmland includes applying a flood damage change detection model to a first remote sensing image and a second remote sensing image of the farmland area to be detected, to obtain a flood damage map of the farmland area to be detected, comprising:

[0014] The first remote sensing image and the second remote sensing image are input into the flood damage change detection model to obtain the flood damage prediction heat map of the farmland area to be detected output by the flood damage change detection model;

[0015] Threshold processing is performed on the predicted flood damage heat map of the farmland area to be monitored to obtain the flood damage map of the farmland area to be monitored.

[0016] According to the present invention, a method for detecting flood-damaged farmland is provided, wherein the flood damage change detection model includes an encoder and a decoder, and the encoder includes a first encoding module, a second encoding module, a third encoding module and a fourth encoding module;

[0017] The step of inputting the first remote sensing image and the second remote sensing image into the flood damage change detection model to obtain the flood damage prediction heat map of the farmland area to be detected output by the flood damage change detection model includes:

[0018] The first remote sensing image and the second remote sensing image are respectively input into the flood damage change detection model. Based on the first coding module in the flood damage change detection model, the first coding feature map corresponding to the first remote sensing image and the second coding feature map corresponding to the second remote sensing image are obtained. Based on the first coding feature map and the second coding feature map, the target first coding feature map is obtained.

[0019] The first encoded feature map and the second encoded feature map are respectively input to the second encoding module to obtain the third encoded feature map corresponding to the first encoded feature map and the fourth encoded feature map corresponding to the second encoded feature map, and the target second encoded feature map is obtained based on the third encoded feature map and the fourth encoded feature map;

[0020] The third coding feature map and the fourth coding feature map are respectively input into the third coding module to obtain the fifth coding feature map corresponding to the third coding feature map and the sixth coding feature map corresponding to the fourth coding feature map, and the target third coding feature map is obtained based on the fifth coding feature map and the sixth coding feature map;

[0021] The fifth and sixth encoded feature maps are respectively input into the fourth encoding module to obtain the seventh encoded feature map corresponding to the fifth encoded feature map and the eighth encoded feature map corresponding to the sixth encoded feature map, and the target fourth encoded feature map is obtained based on the seventh and eighth encoded feature maps;

[0022] Based on the first, second, third, and fourth coding feature maps of the target, the data are applied to the decoder to obtain a predicted heat map of flood damage in the farmland area to be detected.

[0023] The feature dimensions of the first, second, third, and fourth target coding feature maps decrease sequentially.

[0024] According to the present invention, a method for detecting flood-damaged farmland includes a decoder comprising a first decoding module, a second decoding module, a third decoding module, a fourth decoding module, and a classification and segmentation module. The method involves applying the target's first, second, third, and fourth encoded feature maps to the decoder to obtain a predicted heatmap of flood damage in the farmland area to be detected.

[0025] The first decoding module decodes the fourth encoding feature map of the target to obtain a first decoded feature map, and the first decoded feature map of the target is obtained based on the first decoded feature map and the third encoding feature map of the target.

[0026] The second decoding module decodes the first decoding feature map of the target to obtain a second decoding feature map, and the second decoding feature map and the second encoding feature map of the target are used to obtain a third decoding feature map of the target.

[0027] The target third decoding feature map is decoded based on the third decoding module to obtain the third decoding feature map, and the target fourth decoding feature map is obtained based on the third decoding feature map and the target first encoding feature map.

[0028] The fourth decoding module decodes the target fourth decoding feature map to obtain the target fifth decoding feature map. The classification and segmentation module then classifies and segments the target fifth decoding feature map to obtain a flood damage prediction heat map of the farmland area to be detected.

[0029] According to the present invention, a method for detecting flood-damaged farmland includes a semi-supervised training process for a pre-trained flood damage change detection model based on labeled sample image data, unlabeled sample image data, and a flood damage pseudo-label probability map, to obtain a trained flood damage change detection model. The method comprises:

[0030] The pre-trained flood damage change detection model is trained in a supervised manner based on the labeled sample image data, and the supervised loss corresponding to the supervised training is determined.

[0031] Based on the unlabeled sample image data and the flood damage pseudo-label probability map, the pre-trained flood damage change detection model is trained unsupervised using the Top-k strategy to determine the unsupervised loss corresponding to the unsupervised training.

[0032] Based on the supervised loss and the unsupervised loss, the trained flood damage change detection model is obtained.

[0033] According to a method for detecting flood-damaged farmland provided by the present invention, the unsupervised loss is obtained through the following means:

[0034] ;

[0035] in, It is the unsupervised loss of unlabeled sample image data. τ is the batch size of the unlabeled sample image data, and τ is a predefined confidence interval for filtering noise labels. This is a probability map of false labels caused by flood damage in unlabeled sample image data. It is a flood-damaged pseudo-labeled image of unlabeled sample image data, and , The cross-entropy loss is between the flood damage pseudo-label map and the flood damage pseudo-label probability map. The Dis coefficient loss is the difference between the flood damage pseudo-label map and the flood damage pseudo-label probability map.

[0036] The present invention also provides a device for detecting flood-damaged farmland, the device comprising:

[0037] The first flood-damaged farmland detection module is used to acquire the first remote sensing image of the farmland area to be detected before flood damage and the second remote sensing image after flood damage;

[0038] The second flood-damaged farmland detection module is used to apply the first and second remote sensing images of the farmland area to be detected to the flood damage change detection model to obtain the flood damage map of the farmland area to be detected.

[0039] The flood damage change detection model was trained in the following way:

[0040] Constructing a twin deep convolutional neural network;

[0041] The twin deep convolutional neural network is trained based on labeled sample image data to obtain a pre-trained flood damage change detection model. The labeled sample image data includes at least one set of labeled farmland sample image pairs. The labeled farmland sample image pairs include a first farmland sample remote sensing image before flood damage, a second farmland sample remote sensing image after flood damage, and a flood damage real label map corresponding to the second farmland sample remote sensing image.

[0042] Based on the pre-trained flood damage change detection model, the unlabeled sample image data is predicted to obtain a flood damage pseudo-label probability map. The unlabeled sample image data includes at least one set of unlabeled farmland sample image pairs, which include a remote sensing image of the third farmland sample before flood damage and a remote sensing image of the fourth farmland sample after flood damage.

[0043] The pre-trained flood damage change detection model is semi-supervised based on the labeled sample image data, the unlabeled sample image data, and the flood damage pseudo-label probability map to obtain the trained flood damage change detection model.

[0044] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the flood-damaged farmland detection method described above.

[0045] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the flood-damaged farmland detection method as described above.

[0046] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the flood-damaged farmland detection method as described above.

[0047] The present invention provides a method, apparatus, device, and storage medium for detecting flood-damaged farmland, comprising: acquiring a first remote sensing image of the farmland area to be detected before flood damage and a second remote sensing image of the farmland area after flood damage; applying the first and second remote sensing images of the farmland area to be detected to a flood damage change detection model to obtain a flood damage map of the farmland area to be detected; wherein, the flood damage change detection model is trained in the following manner: constructing a Siamese deep convolutional neural network; training the Siamese deep convolutional neural network based on labeled sample image data to obtain a pre-trained flood damage change detection model, wherein the labeled sample image data includes at least one set of labeled farmland sample image pairs, and the labeled farmland sample image pairs include The invention utilizes remote sensing images of a first farmland sample before flood damage, a second farmland sample after flood damage, and a corresponding flood damage real-label map for the second farmland sample. Based on a pre-trained flood damage change detection model, predictions are made on unlabeled sample image data to obtain a flood damage pseudo-label probability map. The unlabeled sample image data includes at least one pair of unlabeled farmland sample images, including a third farmland sample before flood damage and a fourth farmland sample after flood damage. The pre-trained flood damage change detection model is then semi-supervised based on labeled sample image data, unlabeled sample image data, and the flood damage pseudo-label probability map to obtain a trained flood damage change detection model. This invention addresses the problem of scarce labeled datasets of farmland with different degrees of damage in a limited number of flood disaster scenarios. By using semi-supervised training with labeled and unlabeled sample image data, the generalization ability and detection performance of the model are improved. Furthermore, this invention combines pre- and post-disaster remote sensing images to further improve the accuracy of flood-damaged farmland change detection. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0049] Figure 1 This is a flowchart illustrating the method for detecting flood-damaged farmland provided by the present invention;

[0050] Figure 2 This is a schematic diagram of the encoder structure of the flood damage change detection model provided by the present invention;

[0051] Figure 3 This is a schematic diagram of the decoder structure of the flood damage change detection model provided by the present invention;

[0052] Figure 4 This is a schematic diagram of the structure of the flood-damaged farmland detection device provided by the present invention;

[0053] Figure 5 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0055] Figure 1 This is a flowchart illustrating the flood-damaged farmland detection method provided by the present invention, as shown below. Figure 1 As shown, the method includes:

[0056] Step 110: Acquire the first remote sensing image of the farmland area to be detected before flood damage and the second remote sensing image after flood damage;

[0057] In this embodiment, remote sensing images at two different time points are collected, namely, remote sensing images of the farmland area to be detected before and after the flood disaster.

[0058] Here, the first remote sensing image provides the original state information of the farmland area to be detected, which is used for comparison with the second remote sensing image after flood damage.

[0059] Step 120: Based on the first and second remote sensing images of the farmland area to be detected, apply them to the flood damage change detection model to obtain the flood damage map of the farmland area to be detected;

[0060] In this embodiment, the first and second remote sensing images of the farmland area to be detected are input into the flood damage change detection model. The flood damage map of the farmland area to be detected is obtained by the prediction results output by the flood damage change detection model. The flood damage map of the farmland area to be detected is usually represented by different colors or gray values ​​to indicate different degrees of damage.

[0061] The flood damage change detection model was trained in the following way:

[0062] Constructing a twin deep convolutional neural network;

[0063] The twin deep convolutional neural network is trained based on labeled sample image data to obtain a pre-trained flood damage change detection model. The labeled sample image data includes at least one set of labeled farmland sample image pairs. The labeled farmland sample image pairs include a first farmland sample remote sensing image before flood damage, a second farmland sample remote sensing image after flood damage, and a flood damage real label map corresponding to the second farmland sample remote sensing image.

[0064] Based on the pre-trained flood damage change detection model, the unlabeled sample image data is predicted to obtain a flood damage pseudo-label probability map. The unlabeled sample image data includes at least one set of unlabeled farmland sample image pairs, which include a remote sensing image of the third farmland sample before flood damage and a remote sensing image of the fourth farmland sample after flood damage.

[0065] The pre-trained flood damage change detection model is semi-supervised based on the labeled sample image data, the unlabeled sample image data, and the flood damage pseudo-label probability map to obtain the trained flood damage change detection model.

[0066] Here, the Siamese deep convolutional neural network consists of two or more identical sub-networks that share the same weights and parameters. In this embodiment, in the scenario of flood damage detection, the Siamese deep convolutional neural network is used to process remote sensing images before and after the flood separately.

[0067] In this embodiment, the training of the flood damage change detection model is divided into three stages. In the first stage, labeled sample image data containing remote sensing images before and after flood damage and corresponding real flood damage label images are used to pre-train the Siamese deep convolutional neural network. Specifically, the model parameters are optimized based on the loss function between the flood damage prediction label image and the flood damage real label image output by the Siamese deep convolutional neural network to obtain the pre-trained flood damage change detection model.

[0068] Next, in the second stage, the pre-trained flood damage change detection model is used to predict unlabeled sample image data (i.e., only remote sensing images before and after the flood, but without corresponding true flood damage labels), resulting in a flood damage pseudo-label probability map output by the pre-trained flood damage change detection model. Here, the flood damage pseudo-label probability map represents the predicted probability of flood damage at each pixel in the unlabeled sample image data by the pre-trained flood damage change detection model.

[0069] Finally, the pre-trained flood damage change detection model is semi-supervised using labeled sample image data, unlabeled sample image data, and a probability map of flood damage pseudo-labels, resulting in a well-trained flood damage change detection model. Here, semi-supervised training combines supervised and unsupervised training.

[0070] In one example, we will use remote sensing images of the first and third farmland samples before flood damage to illustrate the concept. Specifically, before semi-supervised training, the first and third farmland samples are mixed in the same batch and then shuffled and reversed. This allows the model to learn the high-dimensional features of the first farmland sample image in supervised training, and then learn the high-dimensional features of the third farmland sample image in unsupervised training, thus enhancing the model's generalization ability.

[0071] Furthermore, in this embodiment, before training the model using the first farmland sample remote sensing image and the third farmland sample remote sensing image, the first farmland sample remote sensing image and the third farmland sample remote sensing image can be subjected to enhanced perturbations, such as random cropping, horizontal flipping, vertical flipping, rotation, etc.

[0072] The flood-damaged farmland detection method proposed in this embodiment faces the problem of scarce samples in the labeled dataset of farmland with different degrees of damage under a small number of flood disaster scenarios. By using semi-supervised training, the generalization ability and detection performance of the model are improved. In addition, this invention combines pre-disaster and post-disaster remote sensing images to further improve the detection accuracy of changes in flood-damaged farmland.

[0073] It should be noted that each implementation method of this application can be freely combined, rearranged, or executed individually, and does not need to rely on or depend on a fixed execution order.

[0074] In some embodiments, the application of the first and second remote sensing images of the farmland area to be detected to a flood damage change detection model to obtain a flood damage map of the farmland area to be detected includes:

[0075] The first remote sensing image and the second remote sensing image are input into the flood damage change detection model to obtain the flood damage prediction heat map of the farmland area to be detected output by the flood damage change detection model;

[0076] Threshold processing is performed on the predicted flood damage heat map of the farmland area to be monitored to obtain the flood damage map of the farmland area to be monitored.

[0077] In this embodiment, the first and second remote sensing images are used as inputs and fed into a pre-trained flood damage change detection model. By performing deep learning and feature extraction on these two remote sensing images, the model can identify changes in the farmland area to be detected before and after flooding.

[0078] Here, the flood damage change detection model outputs a predictive heat map, which represents the probability of flood damage in the farmland area to be detected in the form of heat (i.e., color depth or grayscale value).

[0079] Specifically, after obtaining the predicted heat map of flood damage in the farmland area to be monitored, a threshold can be set as needed to classify the pixels in the predicted heat map into different categories, such as "damaged" and "undamaged". Then, the pixels of different categories are represented by different colors or symbols in the final map to obtain the flood damage map of the farmland area to be monitored.

[0080] In some embodiments, the flood damage change detection model includes an encoder and a decoder, wherein the encoder includes a first encoding module, a second encoding module, a third encoding module and a fourth encoding module;

[0081] The step of inputting the first remote sensing image and the second remote sensing image into the flood damage change detection model to obtain the flood damage prediction heat map of the farmland area to be detected output by the flood damage change detection model includes:

[0082] The first remote sensing image and the second remote sensing image are respectively input into the flood damage change detection model. Based on the first coding module in the flood damage change detection model, the first coding feature map corresponding to the first remote sensing image and the second coding feature map corresponding to the second remote sensing image are obtained. Based on the first coding feature map and the second coding feature map, the target first coding feature map is obtained.

[0083] The first encoded feature map and the second encoded feature map are respectively input to the second encoding module to obtain the third encoded feature map corresponding to the first encoded feature map and the fourth encoded feature map corresponding to the second encoded feature map, and the target second encoded feature map is obtained based on the third encoded feature map and the fourth encoded feature map;

[0084] The third coding feature map and the fourth coding feature map are respectively input into the third coding module to obtain the fifth coding feature map corresponding to the third coding feature map and the sixth coding feature map corresponding to the fourth coding feature map, and the target third coding feature map is obtained based on the fifth coding feature map and the sixth coding feature map;

[0085] The fifth and sixth encoded feature maps are respectively input into the fourth encoding module to obtain the seventh encoded feature map corresponding to the fifth encoded feature map and the eighth encoded feature map corresponding to the sixth encoded feature map, and the target fourth encoded feature map is obtained based on the seventh and eighth encoded feature maps;

[0086] Based on the first, second, third, and fourth coding feature maps of the target, the data are applied to the decoder to obtain a predicted heat map of flood damage in the farmland area to be detected.

[0087] The feature dimensions of the first, second, third, and fourth target coding feature maps decrease sequentially.

[0088] refer to Figure 2As shown, the encoder includes two sets of encoding modules. Each set of encoding modules includes a first encoding module, a second encoding module, a third encoding module, and a fourth encoding module. Here, the first encoding module consists of a (conv 4x4, LN) convolutional layer, three ConNeXt Block convolutional blocks, and a downsampled layer. The second encoding module consists of three ConNeXt Block convolutional blocks and a downsampled layer. The third encoding module consists of nine ConNeXt Block convolutional blocks and a downsampled layer. The fourth encoding module consists of three ConNeXt Block convolutional blocks.

[0089] Specifically, in this embodiment, the first encoding module is used to extract features from the first remote sensing image and the second remote sensing image respectively, generating a first encoded feature map corresponding to the first remote sensing image and a second encoded feature map corresponding to the second remote sensing image. Then, based on the first encoded feature map and the second encoded feature map, the target first encoded feature map F1 is obtained through a fusion mechanism.

[0090] Continue to refer to Figure 2 The second, third, and fourth encoding modules repeat a similar process in sequence to obtain the target second encoding feature map F2, the target third encoding feature map F3, and the target fourth encoding feature map F4, whose feature dimensions decrease in sequence.

[0091] In some embodiments, the decoder includes a first decoding module, a second decoding module, a third decoding module, a fourth decoding module, and a classification and segmentation module. The step of applying the target's first encoded feature map, the target's second encoded feature map, the target's third encoded feature map, and the target's fourth encoded feature map to the decoder to obtain a flood damage prediction heatmap of the farmland area to be detected includes:

[0092] The first decoding module decodes the fourth encoding feature map of the target to obtain a first decoded feature map, and the first decoded feature map of the target is obtained based on the first decoded feature map and the third encoding feature map of the target.

[0093] The second decoding module decodes the first decoding feature map of the target to obtain a second decoding feature map, and the second decoding feature map and the second encoding feature map of the target are used to obtain a third decoding feature map of the target.

[0094] The third decoding feature map of the target is decoded based on the second decoding module to obtain the third decoding feature map, and the fourth decoding feature map of the target is obtained based on the third decoding feature map and the first encoding feature map of the target.

[0095] The fourth decoding feature map of the target is decoded based on the second decoding module to obtain the fifth decoding feature map of the target. The fifth decoding feature map of the target is then classified and segmented based on the classification and segmentation module to obtain a flood damage prediction heat map of the farmland area to be detected.

[0096] refer to Figure 3 As shown, the decoder includes a first decoding module, a second decoding module, a third decoding module, a fourth decoding module, and a classification and segmentation module. It should be noted that the first decoding module, the second decoding module, and the third decoding module are all composed of conv 3x3 convolutional layers and upsampled layers, and the fourth decoding module is composed of conv 3x3 convolutional layers.

[0097] In this embodiment, the first decoding module first decodes the target fourth encoded feature map F4 to generate a first decoded feature map. Then, the first decoded feature map is fused with the target third encoded feature map F3 in some form (such as addition, concatenation, or convolution) to obtain the target first decoded feature map. Next, the second decoding module decodes the target first decoded feature map to generate a second decoded feature map. Then, the second decoded feature map is fused with the target second encoded feature map F2 to obtain the target third decoded feature map. This process is repeated until the target fifth decoded feature map is generated. Finally, the target fifth decoded feature map is input into the classification and segmentation module for classification and segmentation to obtain a flood damage prediction heatmap of the farmland area to be detected.

[0098] In this embodiment, the conversion from high-level semantic information to fine-grained spatial information is achieved by progressively decoding and fusing coded feature maps at different levels.

[0099] In some embodiments, the step of semi-supervised training of the pre-trained flood damage change detection model based on the labeled sample image data, the unlabeled sample image data, and the flood damage pseudo-label probability map to obtain the trained flood damage change detection model includes:

[0100] The pre-trained flood damage change detection model is trained in a supervised manner based on the labeled sample image data, and the supervised loss corresponding to the supervised training is determined.

[0101] Based on the unlabeled sample image data and the flood damage pseudo-label probability map, the pre-trained flood damage change detection model is trained unsupervised using the Top-k strategy to determine the unsupervised loss corresponding to the unsupervised training.

[0102] Based on the supervised loss and the unsupervised loss, the trained flood damage change detection model is obtained.

[0103] In this embodiment, during the supervised training process, the pre-trained flood damage change detection model outputs prediction results and compares them with the actual results to obtain a supervised loss that measures the difference between the prediction results of the pre-trained flood damage change detection model and the actual results.

[0104] Specifically, the supervised loss in this embodiment It was obtained in the following way:

[0105] ;

[0106] in, It is the cross-entropy loss between the actual flood damage label map and the predicted flood damage label map of the labeled sample image data. It is the Dis coefficient loss between the true flood damage label map and the predicted flood damage label map of the labeled sample image data.

[0107] Specifically, in this embodiment, It is obtained through the following formula:

[0108] ;

[0109] also, It is obtained through the following formula:

[0110] ;

[0111] in, It is the batch size of the labeled sample image data. It is a real-world label map of flood-damaged sample image data. It is a flood damage prediction label map for labeled sample image data.

[0112] In addition, in this embodiment, when using unlabeled sample image data to perform unsupervised training on the pre-trained flood damage change detection model, the Top-k strategy is used to dynamically select difficult samples from the unlabeled sample image data for unsupervised training.

[0113] Specifically, unsupervised loss is obtained in the following way:

[0114] ;

[0115] in, It is the unsupervised loss of unlabeled sample image data. τ is the batch size of the unlabeled sample image data, and τ is a predefined confidence interval for filtering noise labels. This is a probability map of false labels caused by flood damage in unlabeled sample image data. It is a flood-damaged pseudo-labeled image of unlabeled sample image data, and , The cross-entropy loss is between the flood damage pseudo-label map and the flood damage pseudo-label probability map. The Dissemination coefficient loss between the flood damage pseudo-label map and the flood damage pseudo-label probability map. Refers to and Select the K largest losses.

[0116] Here, The calculation formula is the same as in the above text. The calculation formula is the same. The calculation formula is the same as that in the above text. The calculation formula is the same, so I will not go into details here.

[0117] Finally, there is a loss of oversight. Unsupervised loss Combined, these form a total loss function. :

[0118] ;

[0119] Here, These are preset weighting coefficients.

[0120] In this embodiment, the generalization ability and detection performance of the model are improved by using semi-supervised training with labeled sample image data and unlabeled sample image data.

[0121] Based on any of the above embodiments, the present invention also provides a device for detecting flood-damaged farmland. Figure 4 This is a schematic diagram of the structure of the flood-damaged farmland detection device provided by the present invention, as shown below. Figure 4 As shown, the device includes: a first flood-damaged farmland detection module 410 and a second flood-damaged farmland detection module 420.

[0122] The first flood-damaged farmland detection module 410 is used to acquire the first remote sensing image of the farmland area to be detected before flood damage and the second remote sensing image after flood damage;

[0123] The second flood-damaged farmland detection module 420 is used to apply the first remote sensing image and the second remote sensing image of the farmland area to be detected to the flood damage change detection model to obtain the flood damage map of the farmland area to be detected.

[0124] The flood damage change detection model was trained in the following way:

[0125] Constructing a twin deep convolutional neural network;

[0126] The twin deep convolutional neural network is trained based on labeled sample image data to obtain a pre-trained flood damage change detection model. The labeled sample image data includes at least one set of labeled farmland sample image pairs. The labeled farmland sample image pairs include a first farmland sample remote sensing image before flood damage, a second farmland sample remote sensing image after flood damage, and a flood damage real label map corresponding to the second farmland sample remote sensing image.

[0127] Based on the pre-trained flood damage change detection model, the unlabeled sample image data is predicted to obtain a flood damage pseudo-label probability map. The unlabeled sample image data includes at least one set of unlabeled farmland sample image pairs, which include a remote sensing image of the third farmland sample before flood damage and a remote sensing image of the fourth farmland sample after flood damage.

[0128] The pre-trained flood damage change detection model is semi-supervised based on the labeled sample image data, the unlabeled sample image data, and the flood damage pseudo-label probability map to obtain the trained flood damage change detection model.

[0129] The apparatus provided in this invention improves the generalization ability and detection performance of the model by using semi-supervised training with labeled sample image data and unlabeled sample image data. In addition, this invention further improves the detection accuracy of changes in flood-damaged farmland by combining pre- and post-disaster remote sensing images.

[0130] The flood-damaged farmland detection device described in this embodiment can be referred to in correspondence with the flood-damaged farmland detection method provided by the present invention described above, and will not be described in detail here.

[0131] Figure 5 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 5 As shown, the electronic device may include: a processor 510, a communications interface 520, a memory 530, and a communication bus 540, wherein the processor 510, the communications interface 520, and the memory 530 communicate with each other via the communication bus 540. The processor 510 can call logical instructions in the memory 530 to execute a method for detecting flood-damaged farmland, which includes:

[0132] Acquire the first remote sensing image of the farmland area to be monitored before flood damage and the second remote sensing image after flood damage;

[0133] Based on the first and second remote sensing images of the farmland area to be detected, the flood damage change detection model is applied to obtain the flood damage map of the farmland area to be detected.

[0134] The flood damage change detection model was trained in the following way:

[0135] Constructing a twin deep convolutional neural network;

[0136] The twin deep convolutional neural network is trained based on labeled sample image data to obtain a pre-trained flood damage change detection model. The labeled sample image data includes at least one set of labeled farmland sample image pairs. The labeled farmland sample image pairs include a first farmland sample remote sensing image before flood damage, a second farmland sample remote sensing image after flood damage, and a flood damage real label map corresponding to the second farmland sample remote sensing image.

[0137] Based on the pre-trained flood damage change detection model, the unlabeled sample image data is predicted to obtain a flood damage pseudo-label probability map. The unlabeled sample image data includes at least one set of unlabeled farmland sample image pairs, which include a remote sensing image of the third farmland sample before flood damage and a remote sensing image of the fourth farmland sample after flood damage.

[0138] The pre-trained flood damage change detection model is semi-supervised based on the labeled sample image data, the unlabeled sample image data, and the flood damage pseudo-label probability map to obtain the trained flood damage change detection model.

[0139] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0140] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program that can be stored on a non-transitory computer-readable storage medium, wherein when the computer program is executed by a processor, the computer is able to execute the flood-damaged farmland detection method provided by the above methods, the method comprising:

[0141] Acquire the first remote sensing image of the farmland area to be monitored before flood damage and the second remote sensing image after flood damage;

[0142] Based on the first and second remote sensing images of the farmland area to be detected, the flood damage change detection model is applied to obtain the flood damage map of the farmland area to be detected.

[0143] The flood damage change detection model was trained in the following way:

[0144] Constructing a twin deep convolutional neural network;

[0145] The twin deep convolutional neural network is trained based on labeled sample image data to obtain a pre-trained flood damage change detection model. The labeled sample image data includes at least one set of labeled farmland sample image pairs. The labeled farmland sample image pairs include a first farmland sample remote sensing image before flood damage, a second farmland sample remote sensing image after flood damage, and a flood damage real label map corresponding to the second farmland sample remote sensing image.

[0146] Based on the pre-trained flood damage change detection model, the unlabeled sample image data is predicted to obtain a flood damage pseudo-label probability map. The unlabeled sample image data includes at least one set of unlabeled farmland sample image pairs, which include a remote sensing image of the third farmland sample before flood damage and a remote sensing image of the fourth farmland sample after flood damage.

[0147] The pre-trained flood damage change detection model is semi-supervised based on the labeled sample image data, the unlabeled sample image data, and the flood damage pseudo-label probability map to obtain the trained flood damage change detection model.

[0148] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the flood-damaged farmland detection method provided by the methods described above, the method comprising:

[0149] Acquire the first remote sensing image of the farmland area to be monitored before flood damage and the second remote sensing image after flood damage;

[0150] Based on the first and second remote sensing images of the farmland area to be detected, the flood damage change detection model is applied to obtain the flood damage map of the farmland area to be detected.

[0151] The flood damage change detection model was trained in the following way:

[0152] Constructing a twin deep convolutional neural network;

[0153] The twin deep convolutional neural network is trained based on labeled sample image data to obtain a pre-trained flood damage change detection model. The labeled sample image data includes at least one set of labeled farmland sample image pairs. The labeled farmland sample image pairs include a first farmland sample remote sensing image before flood damage, a second farmland sample remote sensing image after flood damage, and a flood damage real label map corresponding to the second farmland sample remote sensing image.

[0154] Based on the pre-trained flood damage change detection model, the unlabeled sample image data is predicted to obtain a flood damage pseudo-label probability map. The unlabeled sample image data includes at least one set of unlabeled farmland sample image pairs, which include a remote sensing image of the third farmland sample before flood damage and a remote sensing image of the fourth farmland sample after flood damage.

[0155] The pre-trained flood damage change detection model is semi-supervised based on the labeled sample image data, the unlabeled sample image data, and the flood damage pseudo-label probability map to obtain the trained flood damage change detection model.

[0156] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0157] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0158] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical coding feature maps; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting flood-damaged farmland, characterized in that, The method includes: Acquire the first remote sensing image of the farmland area to be monitored before flood damage and the second remote sensing image after flood damage; Based on the first and second remote sensing images of the farmland area to be detected, the flood damage change detection model is applied to obtain the flood damage map of the farmland area to be detected. The flood damage change detection model was trained in the following way: Constructing a twin deep convolutional neural network; The twin deep convolutional neural network is trained based on labeled sample image data to obtain a pre-trained flood damage change detection model. The labeled sample image data includes at least one set of labeled farmland sample image pairs. The labeled farmland sample image pairs include a first farmland sample remote sensing image before flood damage, a second farmland sample remote sensing image after flood damage, and a flood damage real label map corresponding to the second farmland sample remote sensing image. Based on the pre-trained flood damage change detection model, the unlabeled sample image data is predicted to obtain a flood damage pseudo-label probability map. The unlabeled sample image data includes at least one set of unlabeled farmland sample image pairs, which include a remote sensing image of the third farmland sample before flood damage and a remote sensing image of the fourth farmland sample after flood damage. The pre-trained flood damage change detection model is semi-supervised based on the labeled sample image data, the unlabeled sample image data, and the flood damage pseudo-label probability map to obtain the trained flood damage change detection model. Specifically, the pre-trained flood damage change detection model is semi-supervised based on the labeled sample image data, the unlabeled sample image data, and the flood damage pseudo-label probability map to obtain the trained flood damage change detection model, including: The pre-trained flood damage change detection model is trained in a supervised manner based on the labeled sample image data to determine the supervised loss corresponding to the supervised training. The pre-trained flood damage change detection model is then trained unsupervisedly based on the unlabeled sample image data and the flood damage pseudo-label probability map, using a Top-k strategy to determine the unsupervised loss corresponding to the unsupervised training. Finally, the trained flood damage change detection model is obtained based on the supervised loss and the unsupervised loss. The unsupervised loss is obtained in the following way: ； in, It is the unsupervised loss of unlabeled sample image data. τ is the batch size of the unlabeled sample image data, and τ is a predefined confidence interval for filtering noise labels. This is a probability map of false labels caused by flood damage in unlabeled sample image data. It is a flood-damaged pseudo-labeled image of unlabeled sample image data, and , The cross-entropy loss is between the flood damage pseudo-label map and the flood damage pseudo-label probability map. The Dis coefficient loss is the difference between the flood damage pseudo-label map and the flood damage pseudo-label probability map.

2. The method for detecting flood-damaged farmland according to claim 1, characterized in that, The first and second remote sensing images of the farmland area to be detected are applied to the flood damage change detection model to obtain a flood damage map of the farmland area to be detected, including: The first remote sensing image and the second remote sensing image are input into the flood damage change detection model to obtain the flood damage prediction heat map of the farmland area to be detected output by the flood damage change detection model; Threshold processing is performed on the predicted flood damage heat map of the farmland area to be monitored to obtain the flood damage map of the farmland area to be monitored.

3. The method for detecting flood-damaged farmland according to claim 2, characterized in that, The flood damage change detection model includes an encoder and a decoder. The encoder includes a first encoding module, a second encoding module, a third encoding module, and a fourth encoding module. The step of inputting the first remote sensing image and the second remote sensing image into the flood damage change detection model to obtain the flood damage prediction heat map of the farmland area to be detected output by the flood damage change detection model includes: The first remote sensing image and the second remote sensing image are respectively input into the flood damage change detection model. Based on the first coding module in the flood damage change detection model, the first coding feature map corresponding to the first remote sensing image and the second coding feature map corresponding to the second remote sensing image are obtained. Based on the first coding feature map and the second coding feature map, the target first coding feature map is obtained. The first encoded feature map and the second encoded feature map are respectively input to the second encoding module to obtain the third encoded feature map corresponding to the first encoded feature map and the fourth encoded feature map corresponding to the second encoded feature map, and the target second encoded feature map is obtained based on the third encoded feature map and the fourth encoded feature map; The third coding feature map and the fourth coding feature map are respectively input into the third coding module to obtain the fifth coding feature map corresponding to the third coding feature map and the sixth coding feature map corresponding to the fourth coding feature map, and the target third coding feature map is obtained based on the fifth coding feature map and the sixth coding feature map; The fifth and sixth encoded feature maps are respectively input into the fourth encoding module to obtain the seventh encoded feature map corresponding to the fifth encoded feature map and the eighth encoded feature map corresponding to the sixth encoded feature map, and the target fourth encoded feature map is obtained based on the seventh and eighth encoded feature maps; Based on the first, second, third, and fourth coding feature maps of the target, the data are applied to the decoder to obtain a predicted heat map of flood damage in the farmland area to be detected. The feature dimensions of the first, second, third, and fourth target coding feature maps decrease sequentially.

4. The method for detecting flood-damaged farmland according to claim 3, characterized in that, The decoder includes a first decoding module, a second decoding module, a third decoding module, a fourth decoding module, and a classification and segmentation module. The step of applying the target's first encoded feature map, the target's second encoded feature map, the target's third encoded feature map, and the target's fourth encoded feature map to the decoder to obtain a predicted heatmap of flood damage in the farmland area to be detected includes: The first decoding module decodes the fourth encoding feature map of the target to obtain a first decoded feature map, and the first decoded feature map of the target is obtained based on the first decoded feature map and the third encoding feature map of the target. The second decoding module decodes the first decoding feature map of the target to obtain a second decoding feature map, and the second decoding feature map and the second encoding feature map of the target are used to obtain a third decoding feature map of the target. The target third decoding feature map is decoded based on the third decoding module to obtain the third decoding feature map, and the target fourth decoding feature map is obtained based on the third decoding feature map and the target first encoding feature map. The fourth decoding module decodes the target fourth decoding feature map to obtain the target fifth decoding feature map. The classification and segmentation module then classifies and segments the target fifth decoding feature map to obtain a flood damage prediction heat map of the farmland area to be detected.

5. A device for detecting flood-damaged farmland, characterized in that, The device includes: The first flood-damaged farmland detection module is used to acquire the first remote sensing image of the farmland area to be detected before flood damage and the second remote sensing image after flood damage; The second flood-damaged farmland detection module is used to apply the first and second remote sensing images of the farmland area to be detected to the flood damage change detection model to obtain the flood damage map of the farmland area to be detected. The flood damage change detection model was trained in the following way: Constructing a twin deep convolutional neural network; The twin deep convolutional neural network is trained based on labeled sample image data to obtain a pre-trained flood damage change detection model. The labeled sample image data includes at least one set of labeled farmland sample image pairs. The labeled farmland sample image pairs include a first farmland sample remote sensing image before flood damage, a second farmland sample remote sensing image after flood damage, and a flood damage real label map corresponding to the second farmland sample remote sensing image. Based on the pre-trained flood damage change detection model, the unlabeled sample image data is predicted to obtain a flood damage pseudo-label probability map. The unlabeled sample image data includes at least one set of unlabeled farmland sample image pairs, which include a remote sensing image of the third farmland sample before flood damage and a remote sensing image of the fourth farmland sample after flood damage. The pre-trained flood damage change detection model is semi-supervised based on the labeled sample image data, the unlabeled sample image data, and the flood damage pseudo-label probability map to obtain the trained flood damage change detection model. Specifically, the pre-trained flood damage change detection model is semi-supervised based on the labeled sample image data, the unlabeled sample image data, and the flood damage pseudo-label probability map to obtain the trained flood damage change detection model, including: The pre-trained flood damage change detection model is trained in a supervised manner based on the labeled sample image data to determine the supervised loss corresponding to the supervised training. The pre-trained flood damage change detection model is then trained unsupervisedly based on the unlabeled sample image data and the flood damage pseudo-label probability map, using a Top-k strategy to determine the unsupervised loss corresponding to the unsupervised training. Finally, the trained flood damage change detection model is obtained based on the supervised loss and the unsupervised loss. The unsupervised loss is obtained in the following way: ； in, It is the unsupervised loss of unlabeled sample image data. τ is the batch size of the unlabeled sample image data, and τ is a predefined confidence interval for filtering noise labels. This is a probability map of false labels caused by flood damage in unlabeled sample image data. It is a flood-damaged pseudo-labeled image of unlabeled sample image data, and , The cross-entropy loss is between the flood damage pseudo-label map and the flood damage pseudo-label probability map. The Dis coefficient loss is the difference between the flood damage pseudo-label map and the flood damage pseudo-label probability map.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the flood-damaged farmland detection method as described in any one of claims 1 to 4.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the flood-damaged farmland detection method as described in any one of claims 1 to 4.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the flood-damaged farmland detection method as described in any one of claims 1 to 4.

Images