Flood damaged farmland detection method, device and equipment and storage medium

Abstract

The invention provides a flood damaged farmland detection method and device, equipment and a storage medium, and relates to the technical field of image processing, and the method comprises the steps: obtaining a first remote sensing image and a second remote sensing image of a to-be-detected farmland area; based on the first remote sensing image and the second remote sensing image, applying to a flood damage change detection model to obtain a flood damage map, and constructing a twin deep convolutional neural network; performing model training on the twin deep convolutional neural network based on the marked sample image data to obtain a pre-trained flood damage change detection model; predicting the unmarked sample image data based on a pre-trained flood damage change detection model to obtain a flood damage false label probability graph; and performing semi-supervised training on a pre-trained flood damage change detection model based on the marked sample image data, the unmarked sample image data and the flood damage false label probability graph to obtain a trained flood damage change detection model. Therefore, the flood damaged farmland change detection precision is improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method, device, equipment and storage medium for detecting farmland damaged by floods. Background Art

[0002] In the field of remote sensing datasets, there are few scenarios where crops in farmland are affected to varying degrees by floods. The types of assets damaged by floods are complex, and there is a large amount of unlabeled data. Since there are diverse and complex scenarios of damaged farmland after the disaster, automatic change detection is a challenging task.

[0003] At present, there is a lack of detailed monitoring research on farmland with different degrees of damage after disasters. The remote sensing characteristics of farmland with different degrees of flood damage are diverse and complex. Compared with ordinary cultivated land or water bodies, when the damaged farmland is still in a state of accumulation of water, the sediment and debris content is higher. The interaction between flood water bodies and crops and flood water bodies makes the remote sensing characteristics complex. Due to the impact of floods or continuous infiltration of waterlogging, some crops are damaged to varying degrees. For example, sheet and block farmland destroyed by floods; sheet and block farmland damaged by waterlogging. Completely damaged farmland presents dry or wet soil state, and its remote sensing characteristics have obvious differences due to different water content. When crop straw still remains on the farmland, it will lead to more complex spectral, shape, and texture characteristics than completely damaged farmland. The classification geometry of healthy farmland-damaged farmland also further increases the complexity of the segmentation task. Summary of the invention

[0004] The present invention provides a method, device, equipment and storage medium for detecting farmland damaged by floods, so as to solve the defects of the methods for detecting farmland damaged by floods in the prior art.

[0005] The present invention provides a method for detecting farmland damaged by floods, the method comprising: Obtain a first remote sensing image of the farmland area to be detected before the flood damage and a second remote sensing image after the flood damage; Based on the first remote sensing image and the second remote sensing image of the farmland area to be detected, applying the model for detecting changes in flood damage to obtain a flood damage map of the farmland area to be detected; The flood damage change detection model is trained in the following way: Build a twin deep convolutional neural network; Model training is performed on the twin deep convolutional neural network based on the labeled sample image data to obtain a pre-trained flood damage change detection model, wherein the labeled sample image data includes at least one set of labeled farmland sample image pair data, wherein the labeled farmland sample image pair data includes a first farmland sample remote sensing image before flood damage, a second farmland sample remote sensing image after flood damage, and a real flood damage label map corresponding to the second farmland sample remote sensing image; Predicting the unlabeled sample image data based on the pre-trained flood damage change detection model to obtain a flood damage pseudo-label probability map, the unlabeled sample image data includes at least one group of unlabeled farmland sample image pair data, the unlabeled farmland sample image pair data includes a third farmland sample remote sensing image before flood damage and a fourth farmland sample remote sensing image after flood damage; The pre-trained flood damage change detection model is semi-supervised trained based on the labeled sample image data, the unlabeled sample image data and the flood damage pseudo-label probability map to obtain the trained flood damage change detection model.

[0006] According to a method for detecting farmland damaged by floods provided by the present invention, the first remote sensing image and the second remote sensing image based on the farmland area to be detected are applied to a flood damage change detection model to obtain a flood damage map of the farmland area to be detected, including: Inputting the first remote sensing image and the second remote sensing image into a flood damage change detection model to obtain a flood damage prediction heat map of the farmland area to be detected output by the flood damage change detection model; Threshold processing is performed on the flood damage prediction heat map of the farmland area to be detected to obtain a flood damage map of the farmland area to be detected.

[0007] According to a method for detecting flood-damaged farmland provided by the present invention, the flood-damaged change detection model includes an encoder and a decoder, and the encoder includes a first encoding module, a second encoding module, a third encoding module and a fourth encoding module; The step of inputting the first remote sensing image and the second remote sensing image into a flood damage change detection model to obtain a flood damage prediction heat map of the farmland area to be detected output by the flood damage change detection model includes: Inputting the first remote sensing image and the second remote sensing image into a flood damage change detection model respectively, obtaining a first coding feature map corresponding to the first remote sensing image and a second coding feature map corresponding to the second remote sensing image based on the first coding module in the flood damage change detection model, and obtaining a target first coding feature map based on the first coding feature map and the second coding feature map; Inputting the first coding feature map and the second coding feature map into the second coding module respectively, obtaining a third coding feature map corresponding to the first coding feature map and a fourth coding feature map corresponding to the second coding feature map, and obtaining a target second coding feature map based on the third coding feature map and the fourth coding feature map; Inputting the third coding feature map and the fourth coding feature map into the third coding module respectively, obtaining a fifth coding feature map corresponding to the third coding feature map and a sixth coding feature map corresponding to the fourth coding feature map, and obtaining a target third coding feature map based on the fifth coding feature map and the sixth coding feature map; Inputting the fifth coding feature map and the sixth coding feature map into the fourth coding module respectively, obtaining a seventh coding feature map corresponding to the fifth coding feature map and an eighth coding feature map corresponding to the sixth coding feature map, and obtaining a target fourth coding feature map based on the seventh coding feature map and the eighth coding feature map; Based on the target first coding feature map, the target second coding feature map, the target third coding feature map and the target fourth coding feature map, the decoder is applied to obtain a flood damage prediction heat map of the farmland area to be detected; Among them, the feature dimensions of the target first coding feature map, the target second coding feature map, the target third coding feature map, and the target fourth coding feature map decrease in sequence.

[0008] According to a method for detecting flood-damaged farmland provided by the present invention, the decoder includes a first decoding module, a second decoding module, a third decoding module, a fourth decoding module and a classification and segmentation module, and the first target coding feature map, the second target coding feature map, the third target coding feature map and the fourth target coding feature map are applied to the decoder to obtain a flood damage prediction heat map of the farmland area to be detected, including: Decoding the target fourth encoding feature map based on the first decoding module to obtain a first decoding feature map, and obtaining a target first decoding feature map based on the first decoding feature map and the target third encoding feature map; Decoding the target first decoding feature graph based on the second decoding module to obtain a second decoding feature graph, and obtaining a target third decoding feature graph based on the second decoding feature graph and the target second encoding feature graph; Decoding the target third decoding feature map based on the third decoding module to obtain a third decoding feature map, and obtaining a target fourth decoding feature map based on the third decoding feature map and the target first encoding feature map; The fourth decoding feature map of the target is decoded based on the fourth decoding module to obtain the fifth decoding feature map of the target, and the fifth decoding feature map of the target is classified and segmented based on the classification and segmentation module to obtain a flood damage prediction heat map of the farmland area to be detected.

[0009] According to a method for detecting flood-damaged farmland provided by the present invention, semi-supervised training is performed on the pre-trained flood damage change detection model based on the labeled sample image data, the unlabeled sample image data and the flood damage pseudo-label probability map to obtain the trained flood damage change detection model, including: Performing supervised training on the pre-trained flood damage change detection model based on the labeled sample image data, and determining a supervised loss corresponding to the supervised training; Based on the unlabeled sample image data and the flood damage pseudo-label probability map, unsupervised training is performed on the pre-trained flood damage change detection model in combination with the Top-k strategy to determine the unsupervised loss corresponding to the unsupervised training; Based on the supervised loss and the unsupervised loss, a trained flood damage change detection model is obtained.

[0010] According to a method for detecting farmland damaged by floods provided by the present invention, the unsupervised loss is obtained in the following manner: ; in, is the unsupervised loss for unlabeled sample image data, is the batch size of unlabeled sample image data, τ is the predefined confidence interval for filtering noisy labels, is the pseudo-label probability map of flood damage in the unlabeled sample image data, is a pseudo-label map of flood damage in unlabeled sample image data, and , is the cross entropy loss between the flood damage pseudo-label map and the flood damage pseudo-label probability map, is the Diss coefficient loss between the flood damage pseudo label map and the flood damage pseudo label probability map.

[0011] The present invention also provides a device for detecting farmland damaged by floods, the device comprising: The first flood-damaged farmland detection module is used to obtain a first remote sensing image of the farmland area to be detected before the flood damage and a second remote sensing image after the flood damage; A second flood-damaged farmland detection module is used to apply the first remote sensing image and the second remote sensing image of the farmland area to be detected to a flood damage change detection model to obtain a flood damage map of the farmland area to be detected; The flood damage change detection model is trained in the following way: Build a twin deep convolutional neural network; Model training is performed on the twin deep convolutional neural network based on the labeled sample image data to obtain a pre-trained flood damage change detection model, wherein the labeled sample image data includes at least one set of labeled farmland sample image pair data, wherein the labeled farmland sample image pair data includes a first farmland sample remote sensing image before flood damage, a second farmland sample remote sensing image after flood damage, and a real flood damage label map corresponding to the second farmland sample remote sensing image; Predicting the unlabeled sample image data based on the pre-trained flood damage change detection model to obtain a flood damage pseudo-label probability map, the unlabeled sample image data includes at least one group of unlabeled farmland sample image pair data, the unlabeled farmland sample image pair data includes a third farmland sample remote sensing image before flood damage and a fourth farmland sample remote sensing image after flood damage; The pre-trained flood damage change detection model is semi-supervised trained based on the labeled sample image data, the unlabeled sample image data and the flood damage pseudo-label probability map to obtain the trained flood damage change detection model.

[0012] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any of the above-mentioned methods for detecting farmland damaged by floods is implemented.

[0013] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for detecting farmland damaged by floods as described above is implemented.

[0014] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements any of the above-mentioned methods for detecting farmland damaged by floods.

[0015] The method, device, equipment and storage medium for detecting flood-damaged farmland provided by the present invention include: obtaining a first remote sensing image of the farmland area to be detected before the flood damage and a second remote sensing image after the flood damage; based on the first remote sensing image and the second remote sensing image of the farmland area to be detected, applying a flood damage change detection model to obtain a flood damage map of the farmland area to be detected; wherein the flood damage change detection model is obtained by training in the following manner: constructing a twin deep convolutional neural network; model training the twin deep convolutional neural network based on labeled sample image data to obtain a pre-trained flood damage change detection model, wherein the labeled sample image data includes at least one group of labeled farmland sample image pair data, and the labeled farmland sample image pair data includes The first farmland sample remote sensing image before flood damage, the second farmland sample remote sensing image after flood damage, and the real flood damage label map corresponding to the second farmland sample remote sensing image; based on the pre-trained flood damage change detection model, the unlabeled sample image data is predicted to obtain a flood damage pseudo-label probability map, the unlabeled sample image data includes at least one group of unlabeled farmland sample image pair data, and the unlabeled farmland sample image pair data includes the third farmland sample remote sensing image before flood damage and the fourth farmland sample remote sensing image after flood damage; based on the labeled sample image data, the unlabeled sample image data and the flood damage pseudo-label probability map, the pre-trained flood damage change detection model is semi-supervised trained to obtain a trained flood damage change detection model. In this way, the present invention faces the problem of scarcity of labeled data sets of farmlands with different degrees of damage in a small number of flood disaster scenes. The generalization ability and detection performance of the model are improved by using the semi-supervised training method of labeled sample image data and unlabeled sample image data. In addition, the present invention combines remote sensing images before and after the disaster to further improve the detection accuracy of flood-damaged farmland changes. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 It is a schematic diagram of the process of the method for detecting farmland damaged by floods provided by the present invention; Figure 2 It is a structural schematic diagram of an encoder of a flood damage change detection model provided by the present invention; Figure 3 It is a structural schematic diagram of a decoder of the flood damage change detection model provided by the present invention; Figure 4It is a structural schematic diagram of a device for detecting farmland damaged by floods provided by the present invention; Figure 5 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] Figure 1 FIG. 1 is a flow chart of a method for detecting farmland damaged by floods provided by the present invention. Figure 1 As shown, the method includes: Step 110, obtaining a first remote sensing image of the farmland area to be detected before the flood damage and a second remote sensing image after the flood damage; In this embodiment, remote sensing images at two time points are collected respectively, namely, remote sensing images of the farmland area to be detected before and after the flood disaster occurs.

[0020] Here, the first remote sensing image provides the original state information of the farmland area to be inspected, which is used for comparison with the second remote sensing image after flood damage.

[0021] Step 120, applying the first remote sensing image and the second remote sensing image of the farmland area to be detected to a flood damage change detection model to obtain a flood damage map of the farmland area to be detected; In this embodiment, the first remote sensing image and the second remote sensing image of the farmland area to be detected are input into the flood damage change detection model, and the flood damage map of the farmland area to be detected is obtained through the prediction results output by the flood damage change detection model, wherein the flood damage map of the farmland area to be detected usually uses different colors or grayscale values ​​to represent different degrees of damage.

[0022] The flood damage change detection model is trained in the following way: Build a twin deep convolutional neural network; Model training is performed on the twin deep convolutional neural network based on the labeled sample image data to obtain a pre-trained flood damage change detection model, wherein the labeled sample image data includes at least one set of labeled farmland sample image pair data, wherein the labeled farmland sample image pair data includes a first farmland sample remote sensing image before flood damage, a second farmland sample remote sensing image after flood damage, and a real flood damage label map corresponding to the second farmland sample remote sensing image; Predicting the unlabeled sample image data based on the pre-trained flood damage change detection model to obtain a flood damage pseudo-label probability map, the unlabeled sample image data includes at least one group of unlabeled farmland sample image pair data, the unlabeled farmland sample image pair data includes a third farmland sample remote sensing image before flood damage and a fourth farmland sample remote sensing image after flood damage; The pre-trained flood damage change detection model is semi-supervised trained based on the labeled sample image data, the unlabeled sample image data and the flood damage pseudo-label probability map to obtain the trained flood damage change detection model.

[0023] Here, the twin deep convolutional neural network is composed of two or more identical sub-networks that share the same weights and parameters. In this embodiment, in the scenario of flood damage detection, the twin deep convolutional neural network is used to process remote sensing images before and after the flood respectively.

[0024] In this embodiment, the training of the flood damage change detection model is divided into three stages. In the first stage, the twin deep convolutional neural network is pre-trained using labeled sample image data including remote sensing images before and after the flood damage and the corresponding flood damage real label map. Specifically, the model parameters are optimized according to the loss function between the flood damage prediction label map output by the twin deep convolutional neural network and the flood damage real label map to obtain a pre-trained flood damage change detection model.

[0025] Then, in the second stage, the pre-trained flood damage change detection model is used to predict the unlabeled sample image data (i.e., only remote sensing images before and after the flood, but no corresponding flood damage real label map), and the flood damage pseudo-label probability map output by the pre-trained flood damage change detection model is obtained. Here, the flood damage pseudo-label probability map represents the predicted probability of flood damage at each pixel in the unlabeled sample image data by the pre-trained flood damage change detection model.

[0026] Finally, the pre-trained flood damage change detection model is semi-supervised trained using labeled sample image data, unlabeled sample image data, and flood damage pseudo-label probability maps to obtain a trained flood damage change detection model. Here, semi-supervised training combines supervised training with unsupervised training.

[0027] In one example, the first farmland sample remote sensing image and the third farmland sample remote sensing image before flood damage are used as examples for explanation. Specifically, before semi-supervised training, the first farmland sample remote sensing image and the third farmland sample remote sensing image before flood damage are mixed in the same batch and then shuffled and reversed, so that the subsequent model learns the high-dimensional features of the first farmland sample remote sensing image in supervised training, and then learns the high-dimensional features of the third farmland sample remote sensing image through unsupervised training, thereby enhancing the generalization of the model.

[0028] Furthermore, in this embodiment, before using the first farmland sample remote sensing image and the third farmland sample remote sensing image to train the model, the first farmland sample remote sensing image and the third farmland sample remote sensing image can be enhanced and disturbed, such as random cropping, horizontal flipping, vertical flipping, rotation, etc.

[0029] The method for detecting farmland damaged by floods proposed in this embodiment faces the problem of scarcity of labeled data sets of farmland with different degrees of damage in a small number of flood disaster scenarios. The generalization ability and detection performance of the model are improved by using semi-supervised training. In addition, the present invention combines remote sensing images before and after the disaster to further improve the detection accuracy of changes in farmland damaged by floods.

[0030] It should be noted that each implementation method of the present application can be freely combined, the order can be changed, or it can be executed separately, and does not need to rely on or depend on a fixed execution order.

[0031] In some embodiments, the first remote sensing image and the second remote sensing image based on the farmland area to be detected are applied to the flood damage change detection model to obtain the flood damage map of the farmland area to be detected, including: Inputting the first remote sensing image and the second remote sensing image into a flood damage change detection model to obtain a flood damage prediction heat map of the farmland area to be detected output by the flood damage change detection model; Threshold processing is performed on the flood damage prediction heat map of the farmland area to be detected to obtain a flood damage map of the farmland area to be detected.

[0032] In this embodiment, the first remote sensing image and the second remote sensing image are used as inputs to the trained flood damage change detection model. The flood damage change detection model can identify the changes of the farmland area to be detected in the remote sensing image before and after the flood by performing deep learning and feature extraction on the two remote sensing images.

[0033] Here, the output of the flood damage change detection model is a predicted heat map, which represents the probability of flood damage in the farmland area to be detected in the form of heat (i.e., color depth or grayscale value).

[0034] Specifically, after obtaining the flood damage prediction heat map of the farmland area to be detected, a threshold can be set as needed to divide the pixels in the prediction heat map into different categories, such as "damaged" and "undamaged". Then, the pixels of different categories are represented in the final map with different colors or symbols to obtain the flood damage map of the farmland area to be detected.

[0035] In some embodiments, the flood damage change detection model includes an encoder and a decoder, the encoder includes a first encoding module, a second encoding module, a third encoding module, and a fourth encoding module; The step of inputting the first remote sensing image and the second remote sensing image into a flood damage change detection model to obtain a flood damage prediction heat map of the farmland area to be detected output by the flood damage change detection model includes: Inputting the first remote sensing image and the second remote sensing image into a flood damage change detection model respectively, obtaining a first coding feature map corresponding to the first remote sensing image and a second coding feature map corresponding to the second remote sensing image based on the first coding module in the flood damage change detection model, and obtaining a target first coding feature map based on the first coding feature map and the second coding feature map; Inputting the first coding feature map and the second coding feature map into the second coding module respectively, obtaining a third coding feature map corresponding to the first coding feature map and a fourth coding feature map corresponding to the second coding feature map, and obtaining a target second coding feature map based on the third coding feature map and the fourth coding feature map; Inputting the third coding feature map and the fourth coding feature map into the third coding module respectively, obtaining a fifth coding feature map corresponding to the third coding feature map and a sixth coding feature map corresponding to the fourth coding feature map, and obtaining a target third coding feature map based on the fifth coding feature map and the sixth coding feature map; Inputting the fifth coding feature map and the sixth coding feature map into the fourth coding module respectively, obtaining a seventh coding feature map corresponding to the fifth coding feature map and an eighth coding feature map corresponding to the sixth coding feature map, and obtaining a target fourth coding feature map based on the seventh coding feature map and the eighth coding feature map; Based on the target first coding feature map, the target second coding feature map, the target third coding feature map and the target fourth coding feature map, the decoder is applied to obtain a flood damage prediction heat map of the farmland area to be detected; Among them, the feature dimensions of the target first coding feature map, the target second coding feature map, the target third coding feature map, and the target fourth coding feature map decrease in sequence.

[0036] refer to Figure 2 As shown, the encoder includes two groups of encoding modules, each group of encoding modules includes a first encoding module, a second encoding module, a third encoding module and a fourth encoding module. Here, the first encoding module consists of a (conv 4x4, LN) convolution layer, three ConNeXt Block convolution blocks, and a Downsample downsampling layer, the second encoding module consists of three ConNeXtBlock convolution blocks and a Downsample downsampling layer, the third encoding module consists of nine ConNeXt Block convolution blocks and a Downsample downsampling layer, and the fourth encoding module consists of three ConNeXt Block convolution blocks.

[0037] Specifically, in this embodiment, the first encoding module is used to perform feature extraction on the first remote sensing image and the second remote sensing image respectively to generate a first encoding feature map corresponding to the first remote sensing image and a second encoding feature map corresponding to the second remote sensing image. Then, based on the first encoding feature map and the second encoding feature map, a fusion mechanism is used to obtain the target first encoding feature map F1.

[0038] Then continue to refer to Figure 2 The second encoding module, the third encoding module and the fourth encoding module repeat similar processes in sequence to obtain the target second encoding feature map F2, the target third encoding feature map F3, and the target fourth encoding feature map F4 with decreasing feature dimensions in sequence.

[0039] In some embodiments, the decoder includes a first decoding module, a second decoding module, a third decoding module, a fourth decoding module and a classification and segmentation module, and the first target encoding feature map, the second target encoding feature map, the third target encoding feature map and the fourth target encoding feature map are applied to the decoder to obtain a flood damage prediction heat map of the farmland area to be detected, including: Decoding the target fourth encoding feature map based on the first decoding module to obtain a first decoding feature map, and obtaining a target first decoding feature map based on the first decoding feature map and the target third encoding feature map; Decoding the target first decoding feature graph based on the second decoding module to obtain a second decoding feature graph, and obtaining a target third decoding feature graph based on the second decoding feature graph and the target second encoding feature graph; Decoding the target third decoding feature graph based on the second decoding module to obtain a third decoding feature graph, and obtaining a target fourth decoding feature graph based on the third decoding feature graph and the target first encoding feature graph; The fourth decoding feature map of the target is decoded based on the second decoding module to obtain the fifth decoding feature map of the target, and the fifth decoding feature map of the target is classified and segmented based on the classification and segmentation module to obtain a flood damage prediction heat map of the farmland area to be detected.

[0040] refer to Figure 3 As shown, the decoder includes a first decoding module, a second decoding module, a third decoding module, a fourth decoding module and a classification and segmentation module. It should be noted here that the first decoding module, the second decoding module, and the third decoding module are all composed of a conv 3x3 convolutional layer and an Upsample upsampling layer, and the fourth decoding module is composed of a conv 3x3 convolutional layer.

[0041] In this embodiment, the first decoding module is first used to decode the fourth encoding feature map F4 of the target to generate the first decoding feature map. Then, the first decoding feature map is fused with the third encoding feature map F3 of the target in some form (such as addition, concatenation or convolution, etc.) to obtain the first decoding feature map of the target. Next, the second decoding module is used to decode the first decoding feature map of the target to generate the second decoding feature map. Then, the second decoding feature map is fused with the second encoding feature map F2 of the target to obtain the third decoding feature map of the target, and the above similar process is repeated until the fifth decoding feature map of the target is generated. Finally, the fifth decoding feature map of the target is input into the classification and segmentation module for classification and segmentation processing to obtain the flood damage prediction heat map of the farmland area to be detected.

[0042] In this embodiment, the conversion from high-level semantic information to fine spatial information is achieved by gradually decoding and fusing coded feature maps at different levels.

[0043] In some embodiments, the semi-supervised training of the pre-trained flood damage change detection model based on the labeled sample image data, the unlabeled sample image data and the flood damage pseudo-label probability map to obtain the trained flood damage change detection model includes: Performing supervised training on the pre-trained flood damage change detection model based on the labeled sample image data, and determining a supervised loss corresponding to the supervised training; Based on the unlabeled sample image data and the flood damage pseudo-label probability map, unsupervised training is performed on the pre-trained flood damage change detection model in combination with the Top-k strategy to determine the unsupervised loss corresponding to the unsupervised training; Based on the supervised loss and the unsupervised loss, a trained flood damage change detection model is obtained.

[0044] In this embodiment, during the supervised training process, the pre-trained flood damage change detection model outputs a prediction result, and compares it with the actual result to obtain a supervised loss that measures the difference between the prediction result of the pre-trained flood damage change detection model and the actual result.

[0045] Specifically, the supervised loss in this embodiment is It is obtained in the following way: ; in, is the cross entropy loss between the real label map of flood damage and the predicted label map of flood damage of labeled sample image data, It is the Diss coefficient loss between the real label map of flood damage and the predicted label map of flood damage of labeled sample image data.

[0046] Specifically, in this embodiment, It is obtained by the following formula: ; also, It is obtained by the following formula: ; in, is the batch size of labeled sample image data, is the real label map of flood damage for labeled sample image data, It is the flood damage prediction label map of the labeled sample image data.

[0047] In addition, in this embodiment, when unlabeled sample image data is used to perform unsupervised training on the pre-trained flood damage change detection model, a Top-k strategy is used to dynamically select difficult samples from the unlabeled sample image data for unsupervised training.

[0048] Specifically, the unsupervised loss is obtained as follows: ; in, is the unsupervised loss for unlabeled sample image data, is the batch size of unlabeled sample image data, τ is the predefined confidence interval for filtering noisy labels, is the pseudo-label probability map of flood damage in the unlabeled sample image data, is a pseudo-label map of flood damage in unlabeled sample image data, and , is the cross entropy loss between the flood damage pseudo-label map and the flood damage pseudo-label probability map, is the Diss coefficient loss between the flood damage pseudo-label map and the flood damage pseudo-label probability map, Refers to and Select the largest K losses.

[0049] Here, The calculation formula is the same as above The calculation formula is the same as ) is calculated using the same formula as above. The calculation formula is the same as that of , which will not be described in detail here.

[0050] Finally, there is a supervised loss With unsupervised loss Combined to form a total loss function : ; Here, is the preset weight coefficient.

[0051] In this embodiment, the generalization ability and detection performance of the model are improved by using labeled sample image data and unlabeled sample image data for semi-supervised training.

[0052] Based on any of the above embodiments, the present invention further provides a device for detecting farmland damaged by floods. Figure 4 Schematic diagram of the structure of the flood damage farmland detection device provided by the present invention. Figure 4 As shown, the device includes: a first flood-damaged farmland detection module 410 and a second flood-damaged farmland detection module 420.

[0053] The first flood-damaged farmland detection module 410 is used to obtain a first remote sensing image of the farmland area to be detected before the flood damage and a second remote sensing image after the flood damage; A second flood-damaged farmland detection module 420 is used to apply the first remote sensing image and the second remote sensing image of the farmland area to be detected to a flood damage change detection model to obtain a flood damage map of the farmland area to be detected; The flood damage change detection model is trained in the following way: Build a twin deep convolutional neural network; Model training is performed on the twin deep convolutional neural network based on the labeled sample image data to obtain a pre-trained flood damage change detection model, wherein the labeled sample image data includes at least one set of labeled farmland sample image pair data, wherein the labeled farmland sample image pair data includes a first farmland sample remote sensing image before flood damage, a second farmland sample remote sensing image after flood damage, and a real flood damage label map corresponding to the second farmland sample remote sensing image; Predicting the unlabeled sample image data based on the pre-trained flood damage change detection model to obtain a flood damage pseudo-label probability map, the unlabeled sample image data includes at least one group of unlabeled farmland sample image pair data, the unlabeled farmland sample image pair data includes a third farmland sample remote sensing image before flood damage and a fourth farmland sample remote sensing image after flood damage; The pre-trained flood damage change detection model is semi-supervised trained based on the labeled sample image data, the unlabeled sample image data and the flood damage pseudo-label probability map to obtain the trained flood damage change detection model.

[0054] The device provided by the embodiment of the present invention improves the generalization ability and detection performance of the model by using semi-supervised training with labeled sample image data and unlabeled sample image data. In addition, the present invention combines remote sensing images before and after disasters to further improve the accuracy of detecting changes in farmland damaged by floods.

[0055] The device for detecting farmland damaged by floods described in this embodiment and the method for detecting farmland damaged by floods provided by the present invention described above can refer to each other and will not be described in detail here.

[0056] Figure 5 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 5 As shown, the electronic device may include: a processor 510, a communications interface 520, a memory 530 and a communication bus 540, wherein the processor 510, the communications interface 520 and the memory 530 communicate with each other through the communication bus 540. The processor 510 may call the logic instructions in the memory 530 to execute the method for detecting flood-damaged farmland, which includes: Obtain a first remote sensing image of the farmland area to be detected before the flood damage and a second remote sensing image after the flood damage; Based on the first remote sensing image and the second remote sensing image of the farmland area to be detected, applying the model for detecting changes in flood damage to obtain a flood damage map of the farmland area to be detected; The flood damage change detection model is trained in the following way: Build a twin deep convolutional neural network; Model training is performed on the twin deep convolutional neural network based on the labeled sample image data to obtain a pre-trained flood damage change detection model, wherein the labeled sample image data includes at least one set of labeled farmland sample image pair data, wherein the labeled farmland sample image pair data includes a first farmland sample remote sensing image before flood damage, a second farmland sample remote sensing image after flood damage, and a real flood damage label map corresponding to the second farmland sample remote sensing image; Predicting the unlabeled sample image data based on the pre-trained flood damage change detection model to obtain a flood damage pseudo-label probability map, the unlabeled sample image data includes at least one group of unlabeled farmland sample image pair data, the unlabeled farmland sample image pair data includes a third farmland sample remote sensing image before flood damage and a fourth farmland sample remote sensing image after flood damage; The pre-trained flood damage change detection model is semi-supervised trained based on the labeled sample image data, the unlabeled sample image data and the flood damage pseudo-label probability map to obtain the trained flood damage change detection model.

[0057] In addition, the logic instructions in the above-mentioned memory 530 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0058] On the other hand, the present invention further provides a computer program product, the computer program product includes a computer program, the computer program can be stored in a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the flood damage farmland detection method provided by the above methods, the method includes: Obtain a first remote sensing image of the farmland area to be detected before the flood damage and a second remote sensing image after the flood damage; Based on the first remote sensing image and the second remote sensing image of the farmland area to be detected, applying the model for detecting changes in flood damage to obtain a flood damage map of the farmland area to be detected; The flood damage change detection model is trained in the following way: Build a twin deep convolutional neural network; Model training is performed on the twin deep convolutional neural network based on the labeled sample image data to obtain a pre-trained flood damage change detection model, wherein the labeled sample image data includes at least one set of labeled farmland sample image pair data, wherein the labeled farmland sample image pair data includes a first farmland sample remote sensing image before flood damage, a second farmland sample remote sensing image after flood damage, and a real flood damage label map corresponding to the second farmland sample remote sensing image; Predicting the unlabeled sample image data based on the pre-trained flood damage change detection model to obtain a flood damage pseudo-label probability map, the unlabeled sample image data includes at least one group of unlabeled farmland sample image pair data, the unlabeled farmland sample image pair data includes a third farmland sample remote sensing image before flood damage and a fourth farmland sample remote sensing image after flood damage; The pre-trained flood damage change detection model is semi-supervised trained based on the labeled sample image data, the unlabeled sample image data and the flood damage pseudo-label probability map to obtain the trained flood damage change detection model.

[0059] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to execute the flood-damaged farmland detection method provided by the above methods, the method comprising: Obtain a first remote sensing image of the farmland area to be detected before the flood damage and a second remote sensing image after the flood damage; Based on the first remote sensing image and the second remote sensing image of the farmland area to be detected, applying the model for detecting changes in flood damage to obtain a flood damage map of the farmland area to be detected; The flood damage change detection model is trained in the following way: Build a twin deep convolutional neural network; Model training is performed on the twin deep convolutional neural network based on the labeled sample image data to obtain a pre-trained flood damage change detection model, wherein the labeled sample image data includes at least one set of labeled farmland sample image pair data, wherein the labeled farmland sample image pair data includes a first farmland sample remote sensing image before flood damage, a second farmland sample remote sensing image after flood damage, and a real flood damage label map corresponding to the second farmland sample remote sensing image; Predicting the unlabeled sample image data based on the pre-trained flood damage change detection model to obtain a flood damage pseudo-label probability map, the unlabeled sample image data includes at least one group of unlabeled farmland sample image pair data, the unlabeled farmland sample image pair data includes a third farmland sample remote sensing image before flood damage and a fourth farmland sample remote sensing image after flood damage; The pre-trained flood damage change detection model is semi-supervised trained based on the labeled sample image data, the unlabeled sample image data and the flood damage pseudo-label probability map to obtain the trained flood damage change detection model.

[0060] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0061] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0062] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical coding feature diagrams therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting farmland damaged by floods, It is characterized in that The method comprises: Obtain a first remote sensing image of the farmland area to be detected before the flood damage and a second remote sensing image after the flood damage; Based on the first remote sensing image and the second remote sensing image of the farmland area to be detected, applying the model for detecting changes in flood damage to obtain a flood damage map of the farmland area to be detected; The flood damage change detection model is trained in the following way: Build a twin deep convolutional neural network; Model training is performed on the twin deep convolutional neural network based on the labeled sample image data to obtain a pre-trained flood damage change detection model, wherein the labeled sample image data includes at least one set of labeled farmland sample image pair data, wherein the labeled farmland sample image pair data includes a first farmland sample remote sensing image before flood damage, a second farmland sample remote sensing image after flood damage, and a real flood damage label map corresponding to the second farmland sample remote sensing image; Predicting the unlabeled sample image data based on the pre-trained flood damage change detection model to obtain a flood damage pseudo-label probability map, the unlabeled sample image data includes at least one group of unlabeled farmland sample image pair data, the unlabeled farmland sample image pair data includes a third farmland sample remote sensing image before flood damage and a fourth farmland sample remote sensing image after flood damage; The pre-trained flood damage change detection model is semi-supervised trained based on the labeled sample image data, the unlabeled sample image data and the flood damage pseudo-label probability map to obtain the trained flood damage change detection model.

2. The method for detecting farmland damaged by floods according to claim 1, It is characterized in that The first remote sensing image and the second remote sensing image based on the farmland area to be detected are applied to the flood damage change detection model to obtain the flood damage map of the farmland area to be detected, including: Inputting the first remote sensing image and the second remote sensing image into a flood damage change detection model to obtain a flood damage prediction heat map of the farmland area to be detected output by the flood damage change detection model; Threshold processing is performed on the flood damage prediction heat map of the farmland area to be detected to obtain a flood damage map of the farmland area to be detected.

3. The method for detecting farmland damaged by floods according to claim 2, It is characterized in that The flood damage change detection model includes an encoder and a decoder, and the encoder includes a first encoding module, a second encoding module, a third encoding module and a fourth encoding module; The step of inputting the first remote sensing image and the second remote sensing image into a flood damage change detection model to obtain a flood damage prediction heat map of the farmland area to be detected output by the flood damage change detection model includes: Inputting the first remote sensing image and the second remote sensing image into a flood damage change detection model respectively, obtaining a first coding feature map corresponding to the first remote sensing image and a second coding feature map corresponding to the second remote sensing image based on the first coding module in the flood damage change detection model, and obtaining a target first coding feature map based on the first coding feature map and the second coding feature map; Inputting the first coding feature map and the second coding feature map into the second coding module respectively, obtaining a third coding feature map corresponding to the first coding feature map and a fourth coding feature map corresponding to the second coding feature map, and obtaining a target second coding feature map based on the third coding feature map and the fourth coding feature map; Inputting the third coding feature map and the fourth coding feature map into the third coding module respectively, obtaining a fifth coding feature map corresponding to the third coding feature map and a sixth coding feature map corresponding to the fourth coding feature map, and obtaining a target third coding feature map based on the fifth coding feature map and the sixth coding feature map; Inputting the fifth coding feature map and the sixth coding feature map into the fourth coding module respectively, obtaining a seventh coding feature map corresponding to the fifth coding feature map and an eighth coding feature map corresponding to the sixth coding feature map, and obtaining a target fourth coding feature map based on the seventh coding feature map and the eighth coding feature map; Based on the target first coding feature map, the target second coding feature map, the target third coding feature map and the target fourth coding feature map, the decoder is applied to obtain a flood damage prediction heat map of the farmland area to be detected; Among them, the feature dimensions of the target first coding feature map, the target second coding feature map, the target third coding feature map, and the target fourth coding feature map decrease in sequence.

4. The method for detecting farmland damaged by floods according to claim 3, It is characterized in that The decoder includes a first decoding module, a second decoding module, a third decoding module, a fourth decoding module and a classification and segmentation module. The first target coding feature map, the second target coding feature map, the third target coding feature map and the fourth target coding feature map are applied to the decoder to obtain a flood damage prediction heat map of the farmland area to be detected, including: Decoding the target fourth encoding feature map based on the first decoding module to obtain a first decoding feature map, and obtaining a target first decoding feature map based on the first decoding feature map and the target third encoding feature map; Decoding the target first decoding feature graph based on the second decoding module to obtain a second decoding feature graph, and obtaining a target third decoding feature graph based on the second decoding feature graph and the target second encoding feature graph; Decoding the target third decoding feature map based on the third decoding module to obtain a third decoding feature map, and obtaining a target fourth decoding feature map based on the third decoding feature map and the target first encoding feature map; The fourth decoding feature map of the target is decoded based on the fourth decoding module to obtain the fifth decoding feature map of the target, and the fifth decoding feature map of the target is classified and segmented based on the classification and segmentation module to obtain a flood damage prediction heat map of the farmland area to be detected.

5. The method for detecting farmland damaged by floods according to claim 1, It is characterized in that The semi-supervised training of the pre-trained flood damage change detection model based on the labeled sample image data, the unlabeled sample image data and the flood damage pseudo-label probability map to obtain the trained flood damage change detection model includes: Performing supervised training on the pre-trained flood damage change detection model based on the labeled sample image data, and determining a supervised loss corresponding to the supervised training; Based on the unlabeled sample image data and the flood damage pseudo-label probability map, unsupervised training is performed on the pre-trained flood damage change detection model in combination with the Top-k strategy to determine the unsupervised loss corresponding to the unsupervised training; Based on the supervised loss and the unsupervised loss, a trained flood damage change detection model is obtained.

6. The method for detecting farmland damaged by floods according to claim 5, It is characterized in that The unsupervised loss is obtained as follows: ； in, is the unsupervised loss for unlabeled sample image data, is the batch size of unlabeled sample image data, τ is the predefined confidence interval for filtering noisy labels, is the pseudo-label probability map of flood damage in the unlabeled sample image data, is a pseudo-label map of flood damage in unlabeled sample image data, and , is the cross entropy loss between the flood damage pseudo-label map and the flood damage pseudo-label probability map, is the Diss coefficient loss between the flood damage pseudo label map and the flood damage pseudo label probability map.

7. A device for detecting farmland damaged by floods, It is characterized in that The device comprises: The first flood-damaged farmland detection module is used to obtain a first remote sensing image of the farmland area to be detected before the flood damage and a second remote sensing image after the flood damage; A second flood-damaged farmland detection module is used to apply the first remote sensing image and the second remote sensing image of the farmland area to be detected to a flood damage change detection model to obtain a flood damage map of the farmland area to be detected; The flood damage change detection model is trained in the following way: Build a twin deep convolutional neural network; Model training is performed on the twin deep convolutional neural network based on the labeled sample image data to obtain a pre-trained flood damage change detection model, wherein the labeled sample image data includes at least one set of labeled farmland sample image pair data, wherein the labeled farmland sample image pair data includes a first farmland sample remote sensing image before flood damage, a second farmland sample remote sensing image after flood damage, and a real flood damage label map corresponding to the second farmland sample remote sensing image; Predicting the unlabeled sample image data based on the pre-trained flood damage change detection model to obtain a flood damage pseudo-label probability map, the unlabeled sample image data includes at least one group of unlabeled farmland sample image pair data, the unlabeled farmland sample image pair data includes a third farmland sample remote sensing image before flood damage and a fourth farmland sample remote sensing image after flood damage; The pre-trained flood damage change detection model is semi-supervised trained based on the labeled sample image data, the unlabeled sample image data and the flood damage pseudo-label probability map to obtain the trained flood damage change detection model.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, It is characterized in that When the processor executes the program, the method for detecting flood-damaged farmland as described in any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the method for detecting flood-damaged farmland as claimed in any one of claims 1 to 6 is implemented.

10. A computer program product comprising a computer program, It is characterized in that When the computer program is executed by a processor, the method for detecting flood-damaged farmland as claimed in any one of claims 1 to 6 is implemented.

Images