SAR image water body detection method and system based on interpretable deep learning

Abstract

The present invention discloses a SAR image water body detection method and system based on interpretable deep learning, comprising: a backbone network performs feature extraction on a SAR image to obtain low-level and high-level feature maps, and a feature fusion module fuses the low-level and high-level feature maps to obtain a water body feature detection result; a local attribution unit performs local attribution interpretation on the SAR image, outputs an attribution visualization map and an overall positive attribution, uses a ground truth value to mask the overall positive attribution, obtains a specific class positive attribution, and calculates the specific class positive attribution ratio and the positive attribution intersection ratio; a global attribution mapping unit calculates the global specific class positive attribution ratio and the global positive attribution intersection ratio, and ranks all local attribution results to quantify the backbone network performance; a specific class semantic activation mapping module outputs a convolution layer heat map through a high-level feature map for understanding the final attribution. The present invention better compares and explains the feature extraction capability of the backbone network, and more clearly understands the feature attribution through visualization.

Description

Technical Field

[0001] The present invention relates to the field of water body detection, and in particular to a SAR image water body detection method and system based on interpretable deep learning. Background Art

[0002] The use of synthetic aperture radar (SAR) images with all-day, all-weather and cloud-penetrating characteristics to detect surface water has important application value. The water body detection results obtained can timely grasp the changes in floods, water environment evolution, coastline changes, environmental protection, etc., thereby providing important information guidance for national disaster prevention, disaster reduction, rescue, and overall development planning. However, due to the difficulty in reading SAR images, the influence of coherent speckles and the influence of complex backgrounds, high-precision and automatic detection of water bodies faces huge challenges.

[0003] In the past few decades, many scholars have conducted in-depth research on water body detection based on SAR images and made great progress. Most of the traditional methods use geometric models and image processing methods, such as threshold method, clustering method and water body detection algorithm based on boundary extraction. The threshold method classifies pixels with radar backscatter values ​​lower than the given threshold into the "water" category after manually selecting the threshold. However, when the pixel values ​​of different categories are similar, this threshold is difficult to set. The iterative clustering method selects adjacent pixels with similar characteristics to the seed pixel as new seed pixels, and iterates repeatedly until the stop condition is reached. Water body detection methods based on boundary extraction can achieve good results in water body detection in low-resolution SAR images in simple scenes, but these methods are very sensitive to coherent speckle noise. Before water body extraction, the image needs to be preprocessed, which easily causes information loss. In addition, these algorithms do not use the contextual information and semantic information of water body pixels. For high-resolution SAR images with complex background areas, they cannot distinguish water bodies and shadows well when extracting water bodies, and the detection performance is significantly reduced.

[0004] The rapid development of deep learning in recent years has accelerated the research on end-to-end water body detection. For water body detection, the semantic segmentation method in deep learning is widely used. This method is to classify images based on pixels. In water body detection, water bodies such as rivers, lakes and oceans in SAR images are classified as one type of target, while other land scenes are classified as background, which is equivalent to binary pixel-by-pixel classification in semantic segmentation. Using semantic segmentation network for water body detection focuses on higher pixel accuracy and clearer segmentation boundaries. The deep learning network for SAR image water body detection currently proposed is mainly an encoding and decoding network built based on some backbone networks of convolutional neural networks (such as ResNet). Some add some modules to enhance the feature extraction ability of spatial context, such as adding global convolutional network (GCN) and ConvLSTM, and integrating attention mechanism to build corresponding modules; some scholars introduce pyramid pooling construction modules for water body targets of different scales to extract water body features and fuse them; for the problem of rough water body boundaries, some scholars introduce fully convolutional conditional random field (FC-CRF) and local super-resolution restoration technology to improve the precision of water body boundary detection.

[0005] At present, the water body detection method based on convolutional neural network studied by scholars has partially solved the problems of shadow interference and threshold setting in traditional methods, but there are still many challenges, such as the generalization, automation and interpretability of the algorithm. In addition, the existence of inter-class similarities and intra-class differences makes it still difficult to automatically extract water areas from high-resolution SAR images. For example, water bodies and shadows are both shown as extremely low intensity values ​​in SAR images; different types of water bodies such as rivers, isolated ponds or lakes usually reflect low scattering energy, while oceans with large waves reflect high and diverse scattering energy. Surface water bodies include not only large oceans, lakes and wide rivers, but also small water bodies such as streams and ponds. Therefore, in the process of water body extraction, it is necessary to consider not only the macroscopic water body information, but also the edge details of rivers and ponds at the microscopic level, the connectivity of small rivers, etc. This requires the deep learning network to have stronger multi-scale feature extraction capabilities and better feature detail fusion capabilities, and also requires the backbone network to have good performance in the initial feature extraction stage. However, the current deep neural network model (called a "black box") is in an end-to-end learning process. The complexity and hyperparameters of the deep backbone network also make the model difficult to explain, and we cannot intuitively see the network training process.

[0006] In summary, water body detection has made great progress, but there are still great challenges when using only SAR intensity images for detection. SAR imaging is closely related to the electromagnetic scattering characteristics of the target, and water bodies will have specular reflections in SAR images. When the incident angle is not very small, there is almost no echo, so the water body appears dark in the SAR image. The shadow area in the SAR image also has no echo and appears dark; and the intensity value of the road target in the SAR image is very small, which is approximately dark. Without auxiliary information such as DEM, it is difficult to distinguish these targets well. In this regard, many scholars have carried out research and proposed some deep learning networks, which have achieved impressive results. However, the nested nonlinear structure of these networks makes them highly opaque, and it is unclear what information in the input data makes them make decisions and why these results can be achieved. This makes many researchers always skeptical about the use of deep learning in practice, which seriously affects the development and widespread use of deep learning. Therefore, based on the black box characteristics of deep learning networks, the current deep learning systems for water body detection have the following problems:

[0007] (1) The uninterpretability of deep learning networks.

[0008] Although scholars have studied a lot of water extraction networks based on deep learning, the unexplainability of these networks has led to many doubts in their applications. People cannot obtain any information about the model's behavior from the model's structure or weights: whether these designed networks have truly extracted the essential characteristics of water bodies that are different from other targets, what characteristics have been extracted, and why such prediction results are obtained. There is no way to know these, only that a test result is given. This has seriously affected the promotion and application of this technology in practice, and also prevented the value of deep learning from being fully reflected.

[0009] (2) It has little value for the subsequent development of the network.

[0010] Although many deep learning network models can achieve excellent performance, people cannot obtain explanatory results of feature extraction because they cannot interpret deep learning networks. They do not know which part extracts important features, and thus cannot determine which modules and backbone networks are effective parts, making it difficult to effectively optimize existing deep learning networks in a targeted manner to obtain high-performance networks. In addition, people do not know what features deep learning networks have learned, and thus cannot determine which features are decisive for task achievement, which means people cannot learn knowledge from the network and then design modules in a targeted manner to strengthen this feature learning to achieve the goal of further improving network performance.

[0011] (3) It is impossible to know the compatibility of the backbone network and the data.

[0012] The backbone network plays an extremely important role in the feature extraction of data in the deep learning network. A good backbone network can extract rich and recognizable features of the target, and these features are processed by subsequent modules to produce the final result. If the backbone network is not good, no matter how powerful the subsequent modules are, good target extraction results cannot be achieved. In addition, different backbone networks have very different fits for different types of data. For example, for different source data such as daily optical images, optical remote sensing images, and SAR images, the backbone networks suitable for the same target detection or target segmentation may also be very different; even for the same source data (such as SAR images), the performance of the backbone network may be very different for different targets (such as water bodies, overlaps, bridges, airplanes, etc.). However, in the deep learning network, it is impossible to know whether the backbone network is suitable for the data, so the optimal backbone network cannot be selected for the target task. Summary of the invention

[0013] The technical problem to be solved by the present invention is as follows: In view of the above-mentioned problems in the prior art, a method and system for SAR image water body detection based on interpretable deep learning is provided, which can better compare and explain the feature extraction capability of the backbone network and can fully help understand the attribution of backbone network features.

[0014] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0015] A method for detecting water bodies in SAR images based on interpretable deep learning includes selecting the optimal backbone network Backbone for multiple candidate backbone networks using an interpretable deep learning network framework, and the interpretable deep learning network framework includes a water body feature extraction network, a local-global hybrid attribution module LGMA, and a class-specific semantic activation mapping module SSAM. The water body feature extraction network includes a backbone network Backbone and a feature fusion module. The backbone network Backbone is used to perform feature extraction on an input SAR image to obtain a low-level feature map and a high-level feature map. The feature fusion module is used to fuse the low-level feature map and the high-level feature map to obtain a final output SAR image water body feature detection result. The local-global hybrid attribution module LGMA includes a local attribution unit LayerGradCAM and a global attribution mapping unit GAM. The local attribution unit LayerGradCAM is used to perform local attribution interpretation on the input SAR image, and output an attribution visualization map and The overall positive attribution OPA, and then use the label of the ground truth GT to mask the positive attribution obtained by the local attribution explanation to obtain the specific class positive attribution SPA, and calculate the specific class positive attribution proportion PSPA and the positive attribution intersection ratio IOPA of the backbone network according to the overall positive attribution OPA, the specific class positive attribution SPA and the label of the ground truth GT. The global attribution mapping unit GAM is used to calculate the global specific class positive attribution proportion GPSPA and the global positive attribution intersection ratio GIOPA according to the specific class positive attribution proportion PSPA and the positive attribution intersection ratio IOPA of the backbone network, and perform weighted joint ranking on all local attributions including the overall positive attribution OPA and the specific class positive attribution SPA to obtain the global attribution explanation to quantify the detection performance of the backbone network Backbone; the specific class semantic activation shooting module SSAM is used to select the high-level feature map output convolution layer of the water body feature extraction network for heat map visualization, and obtain the focus area of ​​each backbone network Backbone in the water body extraction process for understanding the final attribution.

[0016] Furthermore, the feature fusion module includes a dilated space convolution pooling pyramid module ASPP and a decoding module. The dilated space convolution pooling pyramid module ASPP is used to extract multi-scale context information from high-level feature maps to obtain multi-scale features. The decoding module is used to perform decoding operations on low-level feature maps and multi-scale features to obtain the final output SAR image water feature detection results.

[0017] Furthermore, the decoding module performs decoding operations on the low-level feature map and the multi-scale features to obtain the final output SAR image water feature detection result, including: performing channel dimension reduction on the multi-scale feature through 1×1 convolution, and then performing 4-fold upsampling to obtain the first multi-scale feature; performing 1×1 convolution dimension reduction on the low-level feature map and fusing it with the first multi-scale feature to obtain a fused feature including semantic features and fine-grained features; finally, applying a 3×3 convolution to refine the fused feature and performing bilinear 4-fold upsampling to obtain the final SAR image water feature detection result.

[0018] Furthermore, the local attribution unit LayerGradCAM performs local attribution interpretation on the input SAR image, and the steps of outputting the attribution visualization graph and the overall forward attribution OPA include:

[0019] S1, high-level feature map X output by the backbone network H ∈R 1×N×H×W , in X H In the N feature maps, for the prediction score of a specific class pixel in the i-th row and j-th column, the Argmax function is used to obtain the feature map index value with the largest value, and the feature X containing the index coordinates is returned. max ∈R 1×1×H×W , where N represents the number of feature maps, H, W, C represent the length, width and number of channels of the feature map respectively, and R represents the real number domain;

[0020] S2, according to the high-level feature map X H ∈R 1×N×H×W Create a zero-valued tensor of the same dimension, then add it to X max The index coordinates of the corresponding positions of the corresponding feature maps of the zero-value tensor are supplemented with 1, and the other values ​​are 0, generating binary features containing important feature information indexes, and using binary features to identify the high-level features X H Perform masking to highlight the prediction scores of specific class pixels and focus only on the attribution of specific class pixels;

[0021] S3. For the attribution features after masking, let the specific category score of the pixel in the i-th row and j-th column of any attribution object be category c and be Represents the pixel value of the i-th row and j-th column on the k-th feature map, and the predicted score of each feature map in the masked feature Sum, then Obtain the corresponding gradient by taking the derivative, and then obtain the mean of all pixel gradients on the entire feature map to obtain the weight factor To indicate the importance of the kth feature map, and the importance factor The calculation function expression is:

[0022]

[0023] In the above formula, H and W represent the length and width of the feature map respectively, Z = H × W represents the area of ​​the feature map, and γ represents the pixel semantic index value of the water body pixel. is the classification score of the pixel in the i-th row and j-th column of the attributed object as category c, is the kth feature map;

[0024] S4, the importance factor representing the importance of the k-th feature map Multiply it with the corresponding feature map, and sum all feature maps to generate local explanation attribution, and the function expression for generating local explanation attribution is:

[0025]

[0026] In the above formula, represents local explanation attribution, represents the weight factor representing the importance of the k-th feature map, A k represents the kth feature map;

[0027] S5. For the generated local explanation attribution, the ReLU activation function is used to generate the overall positive attribution OPA, and its function expression is:

[0028] OPA = ReLU (attributions),

[0029] In the above formula, ReLU represents the ReLU activation function, which is used for positive attribution screening; attributes represents local explanation attribution

[0030] Furthermore, the local attribution unit LayerGradCAM uses the label GT to mask the positive attribution obtained by the local attribution interpretation, and obtains the function expression of the specific class positive attribution SPA as follows:

[0031] SPA=ReLU(attributions)∩Pixel(GT) IV＝1 ,

[0032] In the above formula, ReLU represents the ReLU activation function, which is used for positive attribution screening; attributes represents local explanation attribution IV is the index value and uses X IV ＝1 Represents the positive attribution of the area with pixel index value 1 in X, Pixel(GT) IV＝1 Represents the pixels in the water body whose pixel index value is 1 in the label of the ground truth GT, and GT represents the label of the ground truth GT;

[0033] The local attribution unit LayerGradCAM calculates the function expression of the specific class positive attribution proportion PSPA and the positive attribution intersection ratio IOPA of the backbone network according to the overall positive attribution OPA, the specific class positive attribution SPA and the label of the ground truth GT:

[0034]

[0035]

[0036] In the above formula, ReLU represents the ReLU activation function, which is used for positive attribution screening; attributes represents local explanation attribution IV is the index value and uses X IV＝1 Represents the positive attribution of the area with pixel index value 1 in X, Pixel(GT) IV＝1 Represents the pixels in the water body whose pixel index value is 1 in the label of the ground truth GT, and GT represents the label of the ground truth GT;

[0037] Furthermore, the global attribution mapping unit GAM calculates the function expression of the global specific class positive attribution proportion GPSPA and the global positive attribution intersection ratio GIOPA according to the specific class positive attribution proportion PSPA and the positive attribution intersection ratio IOPA of the backbone network:

[0038]

[0039]

[0040] In the above formula, K represents the number of clusters divided in the GAM method, N is the total number of samples, and N i Indicates the number of samples in the i-th cluster, rank(PSPA) i and rank(IOPA) i They are the ranking values ​​of the positive attribution proportion PSPA and the intersection of positive attribution ratio IOPA of the specific class in the i-th cluster.

[0041] Furthermore, the global attribution mapping unit GAM performs weighted joint ranking on all local attributions including the overall positive attribution OPA and the specific class positive attribution SPA to obtain a global attribution explanation including:

[0042] S1, each local attribution vector σ is converted into ω Normalize to get the normalized local attribution vector

[0043]

[0044] In the above formula, σω (i) represents the weighted attribution weight vector of feature i, and the unweighted attribution weight vector of feature i is σ(i), ° represents the Hadamard product operation;

[0045] S2. Calculate the similarity between the normalized local attribution vectors SpearDist(σ,μ) according to the following formula;

[0046]

[0047] In the above formula, σ and μ represent two standardized attribution vectors, μ R (i) and σ R (i) represents the ranking of feature i in the attribution vector μ and σ, respectively. ω (i) and σ ω (i) represents the corresponding ranking μ R (i) and σ R The weight of feature i in (i).

[0048] S3. Group all local attribution vectors. Set K clusters and group each input attribution into the nearest cluster. Update the cluster center by minimizing the pairwise similarity attribution values ​​within the cluster. Repeat multiple iterations to achieve similar attribution grouping. The size of the cluster determines the sophistication of the global attribution explanation. The larger the k value, the more sophisticated the global explanation. Finally, the global attribution explanation is obtained by weighted joint ranking of the importance of attribution features in each cluster.

[0049] Furthermore, the specific class semantic activation mapping module SSAM selects the high-level feature map output convolution layer of the water body feature extraction network to perform heat map visualization, including:

[0050] S1. Extract water body area through semantic segmentation model, where the semantic index value of water body category pixel is 1, the semantic index value of background category pixel is 0, and the semantic category index value to be visualized is specified as 1, and water body feature extraction and visualization are performed;

[0051] S2, extract the input image through the backbone network feature and output the high-level feature X H ∈R 1×N×H×W , establish a two-dimensional zero matrix roi∈R according to the input image H×W , and according to experience, the pixel value of the 0th row and 80th column in the two-dimensional zero matrix roi is set to 1, multiplied by the output high-level feature map, and then the channel sum is returned to a tensor;

[0052] S3. Use back propagation to obtain the weight of each feature map in the sum tensor, and then obtain N weighted feature maps, so as to understand the area that the model focuses on when extracting water bodies. Take the average of the N weighted feature maps to obtain the final features, and finally adjust the negative values ​​to zero through the ReLU function, retaining only the features that have a positive effect on the extraction of water body areas, and obtain the final thermal map after normalization.

[0053] S4. Upsample the heat map to the size of the original input image, and output the final key area display heat map of the backbone network through visualization operation.

[0054] The present invention also includes a SAR image water body detection system based on interpretable deep learning, including a microprocessor and a memory connected to each other, and the microprocessor is programmed or configured to execute the steps of the above-mentioned interpretable deep learning method for SAR image water body detection.

[0055] The present invention also includes a computer-readable storage medium, which stores a computer program for being executed by a computer device to implement the above-mentioned explainable deep learning method for SAR image water body detection.

[0056] Compared with the prior art, the present invention has the following advantages:

[0057] The present invention adopts the Local and Global Mixed Attribution (LGMA) module to realize the attribution analysis of the outputs of different backbone networks in the water body detection framework, thereby providing a very important guidance for the selection of the water body extraction backbone network in the deep learning network.

[0058] The present invention adopts two explanatory evaluation indicators, Specific-class Positive Attribution (SPA) and Overall Positive Attribution (OPA), which can better compare and explain the feature extraction ability of the backbone network.

[0059] The present invention adopts the Semantic Specific-class Activation Mapping (SSAM) module to visualize the water body feature heat map, which more clearly demonstrates the feature attribution of the deep learning network. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 This is a schematic diagram of a SAR image water body detection method based on interpretable deep learning in the present invention.

[0061] Figure 2 It is the water body extraction result and the stage 2 attribution visualization result of the backbone network in water scene I in an embodiment of the present invention.

[0062] Figure 3 This is the water body extraction result and the stage 2 attribution visualization result of the backbone network under water scene II in an embodiment of the present invention.

[0063] Figure 4 This is the water body extraction result and the stage 2 attribution visualization result of the backbone network under water scene III in an embodiment of the present invention.

[0064] Figure 5 This is a visualization display of the SAR image, the corresponding label map, and the output heat map of the four backbone networks in an embodiment of the present invention.

[0065] Figure 6 This is a diagram showing the attribution results of the high-level feature output convolutional layer of the backbone network in an embodiment of the present invention.

[0066] Figure 7 This is a diagram showing the global attribution results of the pond water body in an embodiment of the present invention.

[0067] Figure 8 This is a diagram showing the global attribution results of the river water body in an embodiment of the present invention. DETAILED DESCRIPTION

[0068] The present invention is further described below in conjunction with the accompanying drawings and specific preferred embodiments, but the protection scope of the present invention is not limited thereby.

[0069] like Figure 1As shown, in this embodiment, a SAR image water body detection method based on interpretable deep learning is provided, including using an interpretable deep learning network framework to select the optimal backbone network Backbone for multiple candidate backbone networks Backbone, and the interpretable deep learning network framework includes a water body feature extraction network, a local-global hybrid attribution module LGMA and a specific class semantic activation mapping module SSAM, the water body feature extraction network includes a backbone network Backbone and a feature fusion module, the backbone network Backbone is used to extract features from the input SAR image to obtain a low-level feature map and a high-level feature map, the feature fusion module is used to fuse the low-level feature map and the high-level feature map to obtain the final output SAR image water body feature detection result; the local-global hybrid attribution module LGMA includes a local attribution unit LayerGradCAM and a global attribution mapping unit GAM, the local attribution unit LayerGradCAM is used to perform local attribution interpretation on the input SAR image, and output the attribution map. The visualization graph and the overall positive attribution OPA are used to mask the positive attribution obtained by the local attribution explanation using the label of the ground truth GT to obtain the specific class positive attribution SPA, and the specific class positive attribution proportion PSPA and the positive attribution intersection ratio IOPA of the backbone network are calculated according to the overall positive attribution OPA, the specific class positive attribution SPA and the label of the ground truth GT. The global attribution mapping unit GAM is used to calculate the global specific class positive attribution according to the specific class positive attribution proportion PSPA and the positive attribution intersection ratio IOPA of the backbone network. The attribution proportion GPSPA and the global positive attribution intersection ratio GIOPA, as well as the weighted joint ranking of all local attributions including the overall positive attribution OPA and the specific class positive attribution SPA, are used to obtain the global attribution explanation to quantify the detection performance of the backbone network Backbone; the specific class semantic activation mapping module SSAM is used to select the high-level feature map output convolution layer of the water body feature extraction network for heat map visualization, and obtain the focus area of ​​each backbone network Backbone in the water body extraction process for understanding the final attribution.

[0070] This embodiment uses the Local and Global Mixed Attribution (LGMA) module to implement attribution analysis of the outputs of different backbone networks in the water body detection framework, thereby providing a very important guide for the selection of the water body extraction backbone network in the deep learning network.

[0071] This embodiment uses two explanatory evaluation indicators, Specific-class Positive Attribution (SPA) and Overall Positive Attribution (OPA), which can better compare and explain the feature extraction capability of the backbone network.

[0072] This embodiment uses a semantic specific-class activation mapping (SSAM) module to visualize the water body feature heat map, which more clearly shows the feature attribution of the deep learning network.

[0073] In this embodiment, the feature fusion module includes a dilated space convolution pooling pyramid module ASPP and a decoding module. The dilated space convolution pooling pyramid module ASPP is used to extract multi-scale context information from high-level feature maps to obtain multi-scale features. The decoding module is used to perform decoding operations on low-level feature maps and multi-scale features to obtain the final output SAR image water feature detection results.

[0074] In this embodiment, the step in which the decoding module performs decoding operations on the low-level feature map and the multi-scale features to obtain the final output SAR image water body feature detection result includes: performing channel dimension reduction on the multi-scale feature through 1×1 convolution, and then performing 4-fold upsampling to obtain a first multi-scale feature; performing 1×1 convolution dimension reduction on the low-level feature map and fusing it with the first multi-scale feature to obtain a fused feature including semantic features and fine-grained features; finally, applying a 3×3 convolution to refine the fused feature and performing bilinear 4-fold upsampling to obtain the final SAR image water body feature detection result.

[0075] In this embodiment, the local attribution unit LayerGradCAM performs local attribution interpretation on the input SAR image, and the steps of outputting the attribution visualization graph and the overall forward attribution OPA include:

[0076] S1, high-level feature map X output by the backbone network H ∈R 1×N×H×W , in X H In the N feature maps, for the prediction score of a specific class pixel in the i-th row and j-th column, the Argmax function is used to obtain the feature map index value with the largest value, and the feature X containing the index coordinates is returned. max ∈R 1×1×H×W , where N represents the number of feature maps, H, W, C represent the length, width and number of channels of the feature map respectively, and R represents the real number domain;

[0077] S2, according to the high-level feature map XH ∈R 1×N×H×W Create a zero-valued tensor of the same dimension, then add it to X max The index coordinates of the corresponding positions of the corresponding feature maps of the zero-value tensor are supplemented with 1, and the other values ​​are 0, generating binary features containing important feature information indexes, and using binary features to identify the high-level features X H Masking is to multiply the two features, highlight the prediction scores of specific class pixels, and only focus on the attribution of specific class pixels;

[0078] S3. For the attribution features after masking, let the specific category score of the pixel in the i-th row and j-th column of any attribution object be category c and be Represents the pixel value of the i-th row and j-th column on the k-th feature map, and the predicted score of each feature map in the masked feature Sum, then Obtain the corresponding gradient by taking the derivative, and then obtain the mean of all pixel gradients on the entire feature map to obtain the weight factor To indicate the importance of the kth feature map, and the importance factor The calculation function expression is:

[0079]

[0080] In the above formula, H and W represent the length and width of the feature map respectively, Z = H × W represents the area of ​​the feature map, and γ represents the pixel semantic index value of the water body pixel. is the classification score of the pixel in the i-th row and j-th column of the attributed object as category c, is the kth feature map;

[0081] S4, the importance factor representing the importance of the k-th feature map Multiply it with the corresponding feature map, and sum all feature maps to generate local explanation attribution, and the function expression for generating local explanation attribution is:

[0082]

[0083] In the above formula, represents local explanation attribution, represents the weight factor representing the importance of the k-th feature map, A k represents the kth feature map;

[0084] S5. For the generated local explanation attribution, the ReLU activation function is used to generate the overall positive attribution OPA, and its function expression is:

[0085] OPA=ReLU(attributions), (3)

[0086] In the above formula, ReLU represents the ReLU activation function, which is used for positive attribution screening; attributes represents local explanation attribution

[0087] In this embodiment, the local attribution unit LayerGradCAM uses the label GT to mask the positive attribution obtained by the local attribution interpretation, and obtains the function expression of the specific class positive attribution SPA as follows:

[0088] SPA=ReLU(attributions)∩Pixel(GT) IV＝1 , (4)

[0089] In the above formula, ReLU represents the ReLU activation function, which is used for positive attribution screening; attributes represents local explanation attribution IV is the index value and uses X IV＝1 Represents the positive attribution of the area with pixel index value 1 in X, Pixel(GT) IV＝1 Represents the pixels in the water body whose pixel index value is 1 in the label of the ground truth GT, and GT represents the label of the ground truth GT;

[0090] The local attribution unit LayerGradCAM calculates the function expression of the specific class positive attribution proportion PSPA and the positive attribution intersection ratio IOPA of the backbone network according to the overall positive attribution OPA, the specific class positive attribution SPA and the label of the ground truth GT:

[0091]

[0092]

[0093] In the above formula, RelU represents the ReLU activation function, which is used for positive attribution screening; attributes represents local explanation attribution IV is the index value and uses X IV＝1 Represents the positive attribution of the area with pixel index value 1 in X, Pixel(GT) IV＝1 Represents the pixels in the water body whose pixel index value is 1 in the label of the ground truth GT, and GT represents the label of the ground truth GT;

[0094] In this embodiment, the global attribution mapping unit GAM calculates the function expression of the global specific class positive attribution proportion GPSPA and the global positive attribution intersection ratio GIOPA according to the specific class positive attribution proportion PSPA and the positive attribution intersection ratio IOPA of the backbone network:

[0095]

[0096]

[0097] In the above formula, K represents the number of clusters divided in the GAM method, N is the total number of samples, and N i Indicates the number of samples in the i-th cluster, rank(PSPA) i and rank(IOPA) i They are the ranking values ​​of the positive attribution proportion PSPA and the intersection of positive attribution ratio IOPA of the specific class in the i-th cluster.

[0098] In this embodiment, the global attribution mapping unit GAM performs weighted joint ranking on all local attributions including the overall positive attribution OPA and the specific class positive attribution SPA to obtain the global attribution explanation, including:

[0099] S1, each local attribution vector σ is converted into ω Normalize to get the normalized local attribution vector

[0100]

[0101] In the above formula, σ ω (i) represents the weighted attribution weight vector of feature i, and the unweighted attribution weight vector of feature i is σ(i), ° represents the Hadamard product operation;

[0102] S2. Calculate the similarity between the normalized local attribution vectors SpearDist(σ,μ) according to the following formula;

[0103]

[0104] In the above formula, σ and μ represent two standardized attribution vectors, μ R (i) and σ R (i) represents the ranking of feature i in the attribution vector μ and σ, respectively. ω (i) and σ ω (i) represents the corresponding ranking μ R (i) and σ R The weight of feature i in (i).

[0105] S3. Group all local attribution vectors. Set K clusters and group each input attribution into the nearest cluster. Update the cluster center by minimizing the pairwise similarity attribution values ​​within the cluster. Repeat multiple iterations to achieve similar attribution grouping. The size of the cluster determines the sophistication of the global attribution explanation. The larger the k value, the more sophisticated the global explanation. Finally, the global attribution explanation is obtained by weighted joint ranking of the importance of attribution features in each cluster.

[0106] In this embodiment, the specific class semantic activation shooting module SSAM selects the high-level feature map output convolution layer of the water body feature extraction network to perform heat map visualization, including:

[0107] S1. Extract water body area through semantic segmentation model, where the semantic index value of water body category pixel is 1, the semantic index value of background category pixel is 0, and the semantic category index value to be visualized is specified as 1, and water body feature extraction and visualization are performed;

[0108] S2, extract the input image through the backbone network feature and output the high-level feature X H ∈R 1×N×H×W , establish a two-dimensional zero matrix roi∈R according to the input image H×W , and according to experience, the pixel value of the 0th row and 80th column in the two-dimensional zero matrix roi is set to 1, multiplied by the output high-level feature map, and then the channel sum is returned to a tensor;

[0109] S3. Use back propagation to obtain the weight of each feature map in the sum tensor, and then obtain N weighted feature maps, so as to understand the area that the model focuses on when extracting water bodies. Take the average of the N weighted feature maps to obtain the final features, and finally adjust the negative values ​​to zero through the ReLU function, retaining only the features that have a positive effect on the extraction of water body areas, and obtain the final thermal map after normalization.

[0110] S4. Upsample the heat map to the size of the original input image, and output the final key area display heat map of the backbone network through visualization operation.

[0111] In this embodiment, four representative backbone networks, Mobilenetv2, AlignedXception and ResNet101, are selected to conduct water body extraction experiments, analyze the performance of the representative backbone networks in water body feature extraction, and verify the technical effect of this embodiment by analyzing the output convolution layer attribution visualization results of each backbone network. A brief introduction is given to the four representative backbone networks selected in this embodiment: MobileNetV2 introduces a resource-efficient module with reverse residual and linear bottleneck on the basis of MobileNetV1, which ensures good performance while maintaining lightweight. The deep residual network DRN uses dilated convolution to replace the downsampling layer inside the model on the structure of the residual network, so that the final output layer always maintains high spatial resolution and improves image classification accuracy; and by removing the maximum pooling layer and residual connection and other operations to reduce the "grid artifacts" caused by the dilated convolution, its accuracy advantage is further amplified in applications such as object positioning and semantic segmentation. AlignedXception uses deep separable convolution to achieve a balance between accuracy and speed. ResNet101 has the characteristics of skip connection and optimized residual. It uses a cascade of dilated convolutions with dilation rates of 2, 4, and 8 to replace the last convolution layer, deepen the number of network layers, and enrich the semantic information of feature maps. It is very suitable for building a semantic segmentation network.

[0112] In order to better evaluate the performance of four different backbone networks, this embodiment conducts comparative experiments on the extraction of three water bodies with different characteristics, and selects pixel accuracy (PA), intersection over union (IoU), mean pixel accuracy (MPA) and mean intersection over union (MIoU) as evaluation indicators of water body segmentation results. PA represents the proportion of correctly classified pixels of a certain type of target to the total pixels of that type of target, IoU represents the ratio of the intersection and union of the segmentation results of a certain type of target and the true results of that type of target, MPA represents the mean of the ratio of the number of correctly classified pixels in all categories to the number of all pixels in that category, and MIoU represents the average of IoU over all categories, which is specifically defined as:

[0113]

[0114]

[0115] Where k represents the category of the target, and there are k+1 categories in an image (including one background category); P ij Indicates the number of pixels whose actual type is class i but is predicted to be class j, which is a false positive sample; similarly, Pji P represents the number of pixels whose actual type is class j but is predicted to be class i, which is a false negative sample; ii Indicates the true number of pixels in class i.

[0116] like Figures 2 to 4 The following are the results of water extraction for three scenes with different water characteristics and the attribution visualization results of the backbone network in the second stage. The sizes of the three SAR images are 4000×2000 pixels, 2988×1793 pixels, and 1996×3288 pixels, respectively. (a) is a millimeter wave SAR image; (b) is the label of the water area, blue is the water area, and black is the background; (c)-(f) are the water extraction results based on four different backbone networks, namely MobileNetV2, DRN, AlignedXception, and ResNet101; (g)-(j) are the fusion images of the water extraction results of the four backbone networks and the SAR image, respectively. The yellow box in the figure is the false detection area, that is, the area where the background is mistakenly detected as water; the green box is the missed detection area, that is, the water area that has not been detected; (k)-(n) are the attribution visualization images of the output convolutional layers of the four backbone networks, respectively.

[0117] Depend on Figure 2 (a) It can be seen that the water scene I contains some relatively large pond areas with regular edges and a small river with complex boundaries and some discontinuous water flow. In addition, there are many tree shadows and some roads beside the river in the picture, which can easily cause false detections. Figure 2 (c)-(f), the four backbone networks have good extraction effects on the pond water body with regular boundaries, but AlignedXception does not fully extract the edge of the water body in the river area, and there are also some false detections next to the pond. Figure 2 (g)-(j), the main false detections (yellow boxes) and missed detections (green boxes) are marked. The AlignedXception backbone network has 3 main false detection areas, MobileNetV2 has 1 obvious false detection area and 1 missed detection area, and DRN has only a small area of ​​missed detection. Figure 2 (k)-(n) show the attribution visualization results of the four backbone networks, where the green area represents positive attribution (positive attribution means that the area has a positive effect on the target extraction result). Figure 2(k)-(n) show that although the four backbone networks all give relatively obvious positive attribution of water bodies, the texture features of the water area in the water body positive attribution map output by AlignedXception are very fuzzy, and the ability to extract edge details is relatively poor; the DRN and ResNet101 backbone networks have very obvious contribution attribution values ​​for the water body part (the water body area attribution value is high, and the visual significance of the water body features is obvious), indicating that these two backbone networks have strong ability to extract water body features; among them, DRN has the best water body area attribution value, the attribution map has clear edges, and there is almost no positive attribution in the background area, so there are relatively few false detections in the water body extraction process. Figure 2 This is also verified by the visibility graphs (g)-(j); in the attribution graph (k) of the water area by the MobileNetV2 network, the features of the water area are relatively unclear, the edges are relatively blurred, and some background areas also make positive contributions to water extraction, which can easily cause false detections in the overall process of water extraction.

[0118] Depend on Figure 3 It can be seen that the water area and surrounding features in water scene II are the most complex, with a large number of small ponds and complex edge features. It contains the most edge information and there is no large area of ​​water. In addition, there are shadow areas with slender structures in the image, which increases the complexity of water extraction and the probability of network misjudgment. Figure 3 In (c)-(f), the extraction effect of DRN backbone is also the best, with only one small area missed; ResNet101 has three small areas of missed detection; MobileNetV2 has several missed detections and false detection areas; AlignedXception has a large number of false detections and missed detections and the worst extraction effect. Figure 3 From the attribution visualization of (k)-(n), we can see that the attribution of the MobileNetV2 backbone network Figure 3 There is no obvious edge of the water body in (k), which also shows that the backbone network cannot grasp the details well when extracting small-sized pond water. The water body attribution effect of the AlignedXception backbone network is the worst, and many backgrounds also give strong attribution values, which is very easy to cause false alarms; the attribution effect of the water body area is not obvious, especially the edge part, which is easy to cause missed detection. The attribution results of the ResNet101 and DRN backbone networks can both show obvious edge details. Among them, ResNet101 still has some slightly weak positive attributions in the background area, which has the risk of causing false alarms; while DRN has better attribution of edge details, and there are few positive attributions in the background area. It can be seen from the water body extraction map and the attribution map that DRN has a good extraction effect on the water body area, especially in grasping the edge details.

[0119] Depend on Figure 4(a) It can be seen that, unlike the small-scale water areas in the first two water scenes, the water area in water scene III is mainly composed of a large river running through the image and a lake near a building. The water area is very large. Through experiments on water areas with large scale differences, we can better understand the water body detection performance and robustness of different backbone networks. There are many shadows of buildings, mountains and trees in this water area, which are easy to cause false alarms. Figure 4 (c), (d), (e), (f) Water extraction results of different backbone networks and Figure 4 (b) The comparison shows that the extraction results of DRN and ResNet101 backbone networks are generally good, with no obvious false alarms and missed detections. However, there are many false alarms in the extraction results of AlignedXception, and there are also some small false alarms and missed detections in MobileNetV2. Figure 4 It can be seen from the fusion diagram of (g)-(j). Figure 4 From the attribution visualization diagram (k)-(n), we can see that ResNet101 and DRN backbone networks have obvious positive attribution in the water area, and obvious edge details can be seen, but the positive attribution result of ResNet101 in the small lake area is slightly worse than that of the DRN network, and there are also some slightly smaller positive attribution values ​​in the area around the lake (n). From (k), we can see that the edge details of the water area in the attribution diagram generated by the MobileNetV2 backbone network are not obvious, and some positive attributions appear in the background, which also leads to false alarms of water bodies; the positive attribution results for lake areas are not obvious, which also causes partial missed detection in small water areas; from (m), we can see that in the attribution visualization diagram of the AlignedXception backbone network, the positive attribution is almost spread throughout the entire image, including a large number of background areas, which can easily cause false alarms. This also explains why there are many false alarms in the water extraction results when AlignedXception is used as the backbone network (i). Therefore, this also further verifies that the SAR image water body detection method and system based on interpretable deep learning described in this embodiment can effectively explain the water body extraction of the backbone network.

[0120] In order to analyze the effects of four different backbone networks on water extraction in more detail, Table 1 shows the extraction accuracy of three SAR image scenes containing water areas under different backbone networks. The average extraction accuracy of DRN for water areas in three SAR image scenes is as high as 0.9785, and the average IoU is 0.9547, which are higher than MobileNetV2, ResNet101 and AlignedXception. According to Table 1, for the same water scene, DRN has a very high PA value and the highest IoU value, indicating that DRN can extract water areas more completely for large and small scale waters with very few false alarms; ResNet101 can also extract water bodies more completely, but it has more false alarms than DRN, which can also be seen in the forward attribution map of the backbone network in the three SAR water scenes; AlignedXception has the lowest PA and IoU values, indicating that the backbone network is prone to false alarms and missed detections, which can be explained by the fact that the background in its attribution map also has more positive attribution values. Although the overall extraction effect of MobileNetV2 is acceptable, it is insufficient in learning the details of irregular water areas in small edge areas.

[0121] Table 1 Evaluation indicators of water extraction for four different backbone networks

[0122]

[0123] In order to analyze the performance of the four backbone networks in water feature extraction more deeply and intuitively, this embodiment selects four 512×512 small-scale SAR images containing obvious water areas for heat map visualization analysis. Figure 5 The SAR image, the corresponding label map, and the output heat map visualization results of the four backbone networks are shown respectively. The blue area marked in the label map is the water area with an index value of 1; the black area is the background area with an index value of 0. Figure 5 It can be seen that when MobileNetV2 extracts water bodies, the area of ​​interest includes most of the water area and some background, but the boundary of the water area is not obvious. The response value of AlignedXception to water targets is the lowest among the four backbone networks. At the same time, it includes most of the background targets and cannot accurately distinguish between water bodies and background areas. The final extraction results will also contain a large number of false alarms and missed detections. The output convolutional layer heat maps of the ResNet101 and DRN backbone networks have high response values ​​in the water area, indicating that the two backbone networks pay attention to the water area when extracting water bodies; however, ResNet101's ability to pay attention to edge detail areas is weaker than DRN, such as for Figure 5In image1 and image2, the DRN backbone network can better distinguish the narrow edge areas. The detail information of the convolutional layer heat map it outputs is the most obvious, and there is no redundant background area. This shows that DRN can pay attention to more edge detail information when extracting water bodies, and has excellent extraction capabilities for small ponds, lakes and other water bodies.

[0124] To further explain the water extraction performance of the four backbone networks, this paper also performed attribution visualization analysis on four small-scale SAR images. Figure 6 The attribution results of high-level feature output convolutional layers of the MobileNetV2, DRN, AlignedXception, and ResNet101 backbone networks are shown respectively. Figure 6 It can be seen that the positive attribution response value of the DRN backbone network is relatively high, and they are all distributed in the water area with obvious boundary distinction. ResNet101 has the highest attribution response value, but there are also some slightly lower positive attribution response values ​​in the background area, and the positive attribution of the edge area of ​​the water body is relatively fuzzy, and the details are not obvious enough. The MobileNetV2 network has relatively fewer response values ​​than ResNet101, and the boundaries of the water area are more blurred. The overall response value of AlignedXception is low, and the response values ​​of many water areas are not much different from the background, which is more likely to cause missed detection and false alarm; and the details of the edge of the water body are basically indistinguishable.

[0125] A single example is not sufficient to reflect the overall performance evaluation of each backbone network. Therefore, the GAM algorithm uses independent test samples of 186 large river water bodies and 275 pond water bodies to evaluate the global performance of each backbone network. In the experiment, K = 2 was selected to generate two interpretable clusters. Figure 7 and Figure 8 The global specific class positive attribution percentage (GPSPA) ranking and global intersection of positive attribution ratio (GIOPA) ranking of the four backbone networks for pond and river water bodies are shown respectively, where a(1) and a(2) represent the GPSPA distribution of the four backbone networks for pond water bodies; b(1) and b(2) represent the GIOPA distribution of the four backbone networks for pond water bodies.

[0126] (1) Global attribution analysis of pond water

[0127] Depend on Figure 7As shown in the figure, according to a(1) and a(2), DRN ranks the highest in both clusters of GPSPA distribution. According to Table 2, DRN accounts for 25.77% in GIOPA global ranking cluster b(1), second only to ResNet101, and 30.21% in cluster IIb(2), also closely following ResNet101. ResNet101 has the largest GIOPA ratio, indicating that the water body positive attribution of ResNet101 network contains a more complete water body area, but its GPSPA ratio is the smallest in a(1) and a(2), so the positive attribution of ResNet101 contains some background areas, which may cause some false detections. The GIOPA ratio of AlignedXception is the smallest in both clusters b(1) and b(2), but its GPSPA ratio is relatively high. Therefore, the water body attribution area of ​​AlignedXception contains fewer real water body areas, which is easy to cause missed detections and false detections. Mobilenetv2 ranks third in both clusters of the two indicators, and the overall extraction results are relatively balanced. In summary, DRN has a high and most balanced comprehensive distribution in GPSPA and GIOPA. Therefore, the DRN network has good feature extraction capabilities for small-scale water bodies such as ponds, and is very suitable for building a water body detection network.

[0128] Table 2 Proportion of GAM indicators of pond water bodies by four backbone networks

[0129]

[0130] (2) Global attribution analysis of large river water bodies

[0131] Figure 8 The global ranking results of the percentage of specific positive attributes (GPSPA) and the intersection of positive attributes (GIOPA) for large river water bodies are given.

[0132] Table 3 shows the global rankings of the four backbone networks. Figure 8As shown in (a), DRN ranks highest in both clusters of GPSPA distribution, accounting for 46.41% in cluster I (8 test samples); it accounts for 24.47% in cluster I (126 test samples) and 25.68% in cluster II (60 test samples) in GIOPA global ranking, reaching the average level. ResNet101 ranks second in GPSPA proportion, accounting for 29.84% in cluster I (8 test samples) and 28.34% in cluster II (178 test samples). ResNet101 ranks 27.35% (ranked first) and 29.57% (ranked second) in GIOPA proportion respectively. AlignedXception has the lowest proportion in GIOPA and GPSPA in both clusters. Mobilenetv2 ranks third in both clusters of both indicators, and the overall extraction results are relatively balanced. In general, DRN and ResNet101 have good ability to extract small-scale pond-like water bodies. The attribution contribution of DRN is basically distributed in the water area, and it is less likely to produce false detections.

[0133] Table 3 Proportion of GAM indicators of four backbone networks for large river water bodies

[0134]

[0135] In order to more intuitively understand the overall attribution contribution of the four networks, Table 4 shows the comparison of the index values ​​of the global specific class attribution proportion (GPSPA) and the global positive attribution intersection ratio (GIOPA) of the four backbone networks. As shown in Table 4, the GPSPA and GIOPA of the DRN network are 69.31 and 91.74 respectively, and the difference between the two indicators is the smallest, indicating that the DRN network has a better overall effect on water body extraction and is not prone to false detection; ResNet101 has the highest GIOPA value of 98.48, indicating that the ResNet101 network has a strong contribution to the positive attribution of water bodies, but its GPSPA value is 58.06, and the index difference is relatively large, indicating that the positive attribution of water bodies of the ResNet101 network contains some background areas; the average GPSPA of Mobilenetv2 and AlignedXception is small, 53.12 and 44.65 respectively, but the average GIOPA of Mobilenetv2 is large, 94.74, while the average GIOPA of AlignedXception is only 70.04. The results show that AlignedXception contributes less to feature extraction in the water body detection process. Compared with Mobilenetv2 and AlignedXception, DRN and ResNet101 backbone networks are better able to extract multi-scale water body features.

[0136] Table 1 Comparison of GPSPA and GIOPA indicators of four backbone networks

[0137]

[0138] In order to solve the "black box" problem of deep learning networks, this embodiment combines deep learning networks and interpretability methods to propose an interpretable deep learning method for SAR image water body detection. This method can explain and judge the ability of the backbone network to extract water body features in SAR images, thereby achieving effective selection of the backbone network that is decisive in the water body extraction algorithm. The framework mainly consists of three parts: a water body segmentation network based on four different backbone networks, a local and global attribution module (LGMA), and a specific class semantic activation mapping module (SSAM). The local attribution module attributes the contribution of the output convolutional layer of each backbone network, explicitly observes the performance of each backbone network, and uses the SSAM module to visualize the backbone output heat map. The results show that DRN has better feature extraction performance in the water body extraction process; DRN also shows the best attribution advantage in the global attribution explanation ranking, which further illustrates the feasibility of local attribution visualization. In addition, this paper also uses three millimeter-wave SAR images containing different types of water areas to test each backbone network. The experimental results show that DRN achieves better performance in water body extraction, which further verifies the effectiveness of this paper's framework in explaining deep network models.

[0139] The framework proposed in this paper combines SAR image deep learning with interpretability methods, providing important methodological guidance for the interpretability research of semantic segmentation deep learning networks such as SAR image water body extraction, and inspiring other scholars to further deepen the interpretability research of deep learning networks, so that people can have a deeper understanding of the feature extraction of deep learning networks, thereby making deep learning networks more acceptable to researchers and promoting greater progress and development of SAR image deep learning networks.

[0140] This embodiment also proposes a SAR image water body detection system based on interpretable deep learning, including a microprocessor and a memory connected to each other, and the microprocessor is programmed or configured to execute the steps of the above-mentioned interpretable deep learning method for SAR image water body detection.

[0141] This embodiment also proposes a computer-readable storage medium, which stores a computer program for being executed by a computer device to implement the above-mentioned explainable deep learning method for SAR image water body detection.

[0142] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes. The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, may be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the functions in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the functions specified in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0143] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A SAR image water body detection method based on interpretable deep learning, characterized in that: The invention comprises an explanatory deep learning network framework for selecting the optimal backbone network Backbone for a plurality of candidate backbone networks, and the explanatory deep learning network framework comprises a water body feature extraction network, a local-global hybrid attribution module LGMA and a class-specific semantic activation mapping module SSAM, the water body feature extraction network comprises a backbone network Backbone and a feature fusion module, the backbone network Backbone is used for performing feature extraction on the input SAR image to obtain a low-level feature map and a high-level feature map, the feature fusion module is used for fusing the low-level feature map and the high-level feature map to obtain the final output SAR image water body feature detection result; the local-global hybrid attribution module LGMA comprises a local attribution unit LayerGradCAM and a global attribution mapping unit GAM, the local attribution unit LayerGradCAM is used for performing local attribution interpretation on the input SAR image, outputting an attribution visualization map and an overall forward attribution OPA, and then adopting The positive attribution obtained by the local attribution explanation is masked with the label of the ground truth GT to obtain the specific class positive attribution SPA, and the specific class positive attribution proportion PSPA and the positive attribution intersection ratio IOPA of the backbone network are calculated according to the overall positive attribution OPA, the specific class positive attribution SPA and the label of the ground truth GT. The global attribution mapping unit GAM is used to calculate the global specific class positive attribution proportion GPSPA and the global positive attribution intersection ratio GIOPA according to the specific class positive attribution proportion PSPA and the positive attribution intersection ratio IOPA of the backbone network, and to perform weighted joint ranking on all local attributions including the overall positive attribution OPA and the specific class positive attribution SPA to obtain the global attribution explanation to quantify the detection performance of the backbone network Backbone; the specific class semantic activation mapping module SSAM is used to select the high-level feature map output convolution layer of the water body feature extraction network for heat map visualization, and obtain the focus area of ​​each backbone network Backbone in the water body extraction process for understanding the final attribution.

2. The SAR image water body detection method based on interpretable deep learning according to claim 1 is characterized in that: The feature fusion module includes a dilated space convolution pooling pyramid module ASPP and a decoding module. The dilated space convolution pooling pyramid module ASPP is used to extract multi-scale context information from high-level feature maps to obtain multi-scale features. The decoding module is used to decode low-level feature maps and multi-scale features to obtain the final output SAR image water feature detection result.

3. The SAR image water body detection method based on interpretable deep learning according to claim 2 is characterized in that: The decoding module performs decoding operations on the low-level feature map and the multi-scale features to obtain the final output SAR image water body feature detection result, including: performing channel dimension reduction on the multi-scale feature through 1×1 convolution, and then performing 4-fold upsampling to obtain the first multi-scale feature; performing 1×1 convolution dimension reduction on the low-level feature map and fusing it with the first multi-scale feature to obtain a fused feature including semantic features and fine-grained features; finally, applying a 3×3 convolution to refine the fused feature and performing bilinear 4-fold upsampling to obtain the final SAR image water body feature detection result.

4. The SAR image water body detection method based on interpretable deep learning according to claim 3 is characterized in that: The local attribution unit LayerGradCAM performs local attribution interpretation on the input SAR image, and the steps of outputting the attribution visualization graph and the overall forward attribution OPA include: S1, high-level feature map X output by the backbone network H ∈R 1×N×H×W , in X H In the N feature maps, for the prediction score of a specific class pixel in the i-th row and j-th column, the Argmax function is used to obtain the feature map index value with the largest value, and the feature X containing the index coordinates is returned. max ∈R 1×1×H×W , where N represents the number of feature maps, H, W, C represent the length, width and number of channels of the feature map respectively, and R represents the real number domain; S2, according to the high-level feature map X H ∈R 1×N×H×W Create a zero-valued tensor of the same dimension, then add it to X max The index coordinates of the corresponding positions of the corresponding feature maps of the zero-value tensor are supplemented with 1, and the other values ​​are 0, generating binary features containing important feature information indexes, and using binary features to identify the high-level features X H Perform masking to highlight the prediction scores of specific class pixels and focus only on the attribution of specific class pixels; S3. For the attribution features after masking, let the specific category score of the pixel in the i-th row and j-th column of any attribution object be category c and be Represents the pixel value of the i-th row and j-th column on the k-th feature map, and the predicted score of each feature map in the masked feature Sum, then Obtain the corresponding gradient by taking the derivative, and then obtain the mean of all pixel gradients on the entire feature map to obtain the weight factor To indicate the importance of the kth feature map, and the importance factor The calculation function expression is: In the above formula, H and W represent the length and width of the feature map respectively, Z = H × W represents the area of ​​the feature map, and γ represents the pixel semantic index value of the water body pixel. is the classification score of the pixel in the i-th row and j-th column of the attributed object as category c, is the kth feature map; S4, the importance factor representing the importance of the k-th feature map Multiply it with the corresponding feature map, and sum all feature maps to generate local explanation attribution, and the function expression for generating local explanation attribution is: In the above formula, represents local explanation attribution, represents the weight factor representing the importance of the k-th feature map, A k represents the kth feature map; S5. For the generated local explanation attribution, the ReLU activation function is used to generate the overall positive attribution OPA, and its function expression is: OPA = ReLU (attributions), In the above formula, ReLU represents the ReLU activation function, which is used for positive attribution screening; attributes represents local explanation attribution 5. The SAR image water body detection method based on interpretable deep learning according to claim 4 is characterized in that: The local attribution unit LayerGradCAM uses the label GT to mask the positive attribution obtained by the local attribution interpretation, and obtains the function expression of the specific class positive attribution SPA as follows: SPA=ReLU(attributions)∩Pixel(GT) IV＝1 , In the above formula, ReLU represents the ReLU activation function, which is used for positive attribution screening; attributes represents the local explanation attribution L c LayerGrad-CAM , IV is the index value, and X is used IV＝1 Represents the positive attribution of the area with pixel index value 1 in X, Pixel(GT) IV＝1 Represents the pixels in the water body whose pixel index value is 1 in the label of the ground truth GT, and GT represents the label of the ground truth GT; The local attribution unit LayerGradCAM calculates the function expression of the specific class positive attribution proportion PSPA and the positive attribution intersection ratio IOPA of the backbone network according to the overall positive attribution OPA, the specific class positive attribution SPA and the label of the ground truth GT: In the above formula, ReLU represents the ReLU activation function, which is used for positive attribution screening; attributes represents local explanation attribution IV is the index value and uses X IV＝1 Represents the positive attribution of the area with pixel index value 1 in X, Pixel(GT) IV＝1 The pixels in the water body with pixel index value 1 in the label of the ground truth GT are represented, and GT represents the label of the ground truth GT.

6. The SAR image water body detection method based on interpretable deep learning according to claim 5 is characterized in that: The global attribution mapping unit GAM calculates the global specific class positive attribution proportion GPSPA and the global positive attribution intersection ratio GIOPA according to the specific class positive attribution proportion PSPA and the positive attribution intersection ratio IOPA of the backbone network. The functional expression is: In the above formula, K represents the number of clusters divided in the GAM method, N is the total number of samples, and N i Indicates the number of samples in the i-th cluster, rank(PSPA) i and rank(IOPA) i They are the ranking values ​​of the positive attribution proportion PSPA and the intersection of positive attribution ratio IOPA of the specific class in the i-th cluster.

7. The SAR image water body detection method based on interpretable deep learning according to claim 6 is characterized in that: The global attribution mapping unit GAM performs weighted joint ranking on all local attributions including the overall positive attribution OPA and the specific class positive attribution SPA to obtain the global attribution explanation, including: S1, each local attribution vector σ is converted into ω Normalize to get the normalized local attribution vector In the above formula, σ ω (i) represents the weighted attribution weight vector of feature i, and the unweighted attribution weight vector of feature i is σ(i), Represents the Hadamard product operation; S2. Calculate the similarity between the normalized local attribution vectors SpearDist(σ,μ) according to the following formula; In the above formula, σ and μ represent two standardized attribution vectors, μ R (i) and σ R (i) represents the ranking of feature i in the attribution vector μ and σ, respectively. ω (i) and σ ω (i) represents the corresponding ranking μ R (i) and σ R The weight of feature i in (i); S3. Group all local attribution vectors. Set K clusters and group each input attribution into the nearest cluster. Update the cluster center by minimizing the pairwise similarity attribution values ​​within the cluster. Repeat multiple iterations to achieve similar attribution grouping. The size of the cluster determines the sophistication of the global attribution explanation. The larger the k value, the more sophisticated the global explanation. Finally, the global attribution explanation is obtained by weighted joint ranking of the importance of attribution features in each cluster.

8. The SAR image water body detection method based on interpretable deep learning according to claim 7 is characterized in that: The specific class semantic activation mapping module SSAM selects the high-level feature map output convolution layer of the water body feature extraction network to perform heat map visualization, including: S1. Extract water body area through semantic segmentation model, where the semantic index value of water body category pixel is 1, the semantic index value of background category pixel is 0, and the semantic category index value to be visualized is specified as 1, and water body feature extraction and visualization are performed; S2, extract the input image through the backbone network feature and output the high-level feature X H ∈R 1×N×H×W , establish a two-dimensional zero matrix roi∈R according to the input image H×W , and according to experience, the pixel value of the 0th row and 80th column in the two-dimensional zero matrix roi is set to 1, multiplied by the output high-level feature map, and then the channel sum is returned to a tensor; S3. Use back propagation to obtain the weight of each feature map in the sum tensor, and then obtain N weighted feature maps, so as to understand the area that the model focuses on when extracting water bodies. Take the average of the N weighted feature maps to obtain the final features. Finally, adjust the negative values ​​to zero through the ReLU function, and only retain the features that have a positive effect on the extraction of water body areas. Normalize the process to obtain the final thermal map. S4. Upsample the heat map to the size of the original input image, and output the final key area display heat map of the backbone network through visualization operation.

9. A SAR image water body detection system based on interpretable deep learning, comprising a microprocessor and a memory connected to each other, characterized in that: The microprocessor is programmed or configured to execute the steps of the SAR image water body detection method based on interpretable deep learning as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program for being executed by a computer device to implement the SAR image water body detection method based on interpretable deep learning as described in any one of claims 1 to 8.

Images