Fully polarimetric SAR image segmentation network model, segmentation method and training method

Through the multi-scale feature extraction and fusion mechanism of the fully polarized SAR image segmentation network model, the existing SAR sea ice classification methods in noise impact and context information utilization are solved, and classification accuracy and semantic information capture capabilities are improved.

CN118657946BActive Publication Date: 2024-11-12AEROSPACE INFORMATION RES INST CAS
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411134201.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-19
Publication Date
2024-11-12
Estimated Expiration
2044-08-19

AI Technical Summary

Technical Problem

The existing SAR sea ice classification method based on deep learning is susceptible to image noise during image segmentation and cannot fully utilize image context information, resulting in low classification accuracy.

Method used

The fully polarized SAR image segmentation network model is adopted, and multi-scale feature extraction and fusion is performed through the first branch, the second branch and the third branch in parallel, combining hollow convolution, multi-scale dynamic convolution and attention mechanism. Capture different sizes and semantic information of the image, and realize the effective fusion of feature maps through the weighted fusion module.

Benefits of technology

It improves the accuracy of image segmentation, enhances the ability to capture multi-scale semantic information on the image, reduces the sensitivity to noise, and makes full use of image context information.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118657946B_ABST
    Figure CN118657946B_ABST
Patent Text Reader

Abstract

The present invention provides a full-polarization SAR image segmentation network model, a segmentation method and a training method, which relate to the field of satellite remote sensing, and include: a first branch, which is used to perform multiple residual operations on an image to obtain first feature maps of different dimensions, capture information of different sizes in the first feature map output by the last residual operation based on hole convolution and multi-scale dynamic convolution, obtain a second feature map, fuse the two types of feature maps and then perform convolution to obtain a first fused feature map; a second branch, which is used to perform operations on the image based on self-attention, and then perform convolution on second feature maps of different dimensions obtained by fusing space and channel attention to obtain a second fused feature map; a third branch, which is used to fuse the first feature map and the second feature map, and then splice and convolve on multiple third fused feature maps obtained by capturing information of different sizes in the third fused feature maps based on hole convolution and multi-scale dynamic convolution to obtain a fourth fused feature map; and a weighted fusion module, which is used to fuse the outputs of the three branches.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, specifically to the field of satellite remote sensing technology and the field of deep learning technology, and in particular to a fully polarized SAR image segmentation network model, a segmentation method and a training method. Background Art

[0002] With the development of science and technology, airborne and satellite-borne remote sensing technologies have gradually become efficient means of sea ice monitoring, with advantages mainly in large-scale, sustainable, real-time, and low-cost. Synthetic Aperture Radar (SAR) is not affected by clouds and weather, and can achieve all-day and all-weather observations. At the same time, it can provide images of various resolutions and widths through different working modes to meet the needs of different scenarios, and is an important means of sea ice observation.

[0003] At present, in the field of SAR sea ice classification based on deep learning methods, most of the sea ice classification problems are solved through visual interpretation and annotation, which is not very reliable and leads to the selection of training samples being not objective enough. Moreover, these classification methods are all based on image recognition, that is, a small piece of sea ice is cut out from the entire sea ice image and sent to the network to judge the category mode. This method cannot make full use of the contextual information in the image and is easily affected by the speckle noise in the image. Different areas of the same sea ice may show different scattering intensities, which can easily be misjudged as different categories. Summary of the invention

[0004] In view of this, the present invention provides a full-polarization SAR image segmentation network model, a segmentation method and a training method, which are used to at least partially solve the above technical problems.

[0005] One aspect of the present invention provides a full polarization SAR image segmentation network model, comprising: a first branch, a second branch and a third branch connected in parallel, and a weighted fusion module; the first branch is configured to perform multiple residual operations on the full polarization SAR image to obtain multiple first feature maps of different dimensions, based on the hole convolution and the multi-scale dynamic convolution to capture the information of different sizes in the first feature map output by the last residual operation to obtain the second feature map, and the second feature map is fused with the other first feature maps of different dimensions and then convolved to obtain the first fused feature map; wherein the first feature map output by the previous residual operation is used as the input of the next residual operation; the second branch is configured to perform multiple operations on the full polarization SAR image based on the self-attention mechanism to obtain multiple second feature maps of different dimensions, based on the spatial attention mechanism and the channel attention mechanism, the multiple second feature maps of different dimensions are fused and then convolved to obtain a second fused feature map, wherein the second feature map outputted by the previous operation is used as the input of the next operation; the third branch is configured to fuse the first feature map and the corresponding second feature map respectively to obtain multiple third fused feature maps, based on the hole convolution and the multi-scale dynamic convolution, the information of different sizes in the multiple third fused feature maps is captured to obtain multiple third feature maps, the multiple third feature maps are spliced ​​and convolved to obtain a fourth fused feature map; the weighted fusion module is configured to weightedly fuse the first fused feature map, the second fused feature map, and the fourth fused feature map to obtain the image segmentation result.

[0006] According to an embodiment of the present invention, the first branch includes at least two residual blocks, a multi-scale dynamic attention module, a first decoding module and a convolution layer connected in series in sequence; the at least two residual blocks connected in series in sequence are configured to perform multiple residual operations on the fully polarized SAR image to obtain multiple first feature maps of different dimensions, wherein the output of the previous residual operation module is used as the input of the next residual operation module, the multi-scale dynamic attention module is configured to capture information of different sizes in the first feature map output by the last residual operation based on hole convolution and multi-scale dynamic convolution to obtain a second feature map, the first decoding module and the convolution layer are configured to fuse the second feature map with the remaining first feature maps of different dimensions and then perform convolution processing to obtain a first fused feature map; the second branch includes a self-attention operation module, a second decoding module and a convolution layer connected in series in sequence, which are equal in number to the residual blocks. convolution layer; at least two residual blocks connected in series are configured to perform multiple operations on the fully polarized SAR image to obtain multiple second feature maps of different dimensions; the second decoding module and the convolution layer are configured to fuse the multiple second feature maps of different dimensions based on the spatial attention mechanism and the channel attention mechanism and then perform convolution processing to obtain a second fused feature map; the third branch includes fusion modules, multi-scale dynamic attention modules, first decoding modules and convolution layers, which are equal to the number of residual blocks; each fusion module is configured to fuse a first feature map and a corresponding second feature map; the multi-scale dynamic attention module is configured to capture information of different sizes in multiple third fused feature maps based on void convolution and multi-scale dynamic convolution; the first decoding module and the convolution layer splice and convolve the output of the multi-scale dynamic attention module to obtain a third fused feature map.

[0007] According to an embodiment of the present invention, the multi-scale dynamic attention module includes: a hole convolution pooling unit, a multi-scale extraction unit, a splicing unit, a convolution unit and a channel attention unit; the hole convolution pooling unit is configured to perform different expansion rate processing, convolution processing and pooling processing on the first feature map or the third fused feature map to extract and fuse information of different sizes; the multi-scale extraction unit is configured to extract and fuse information of multi-scale and different receptive fields of the first feature map or the third fused feature map; the splicing unit is configured to splice the first feature map or the third fused feature map with the corresponding output of the hole convolution pooling unit and the multi-scale extraction unit to obtain a spliced ​​feature map; the convolution unit is configured to perform convolution processing on the spliced ​​feature map to reduce the channel dimension; the channel attention unit is configured to obtain the weight corresponding to each channel through global pooling and a fully connected layer, and then multiply it channel by channel with the spliced ​​feature map after dimensionality reduction to obtain the second feature map corresponding to the first feature map or the third feature map corresponding to the third fused feature map.

[0008] According to an embodiment of the present invention, each scale extraction unit in the multi-scale extraction unit includes two extraction branches, one extraction branch is configured to perform dimensionality reduction on the first feature map or the third fused feature map, and the other extraction branch is configured to perform pooling and dimensionality reduction on the first feature map or the third fused feature map, and then the outputs of the two extraction branches are fused based on depthwise separable convolution.

[0009] According to an embodiment of the present invention, different expansion rates include 3, 6, and 9.

[0010] According to an embodiment of the present invention, the dimension of the first feature map is twice the dimension of the second feature map; the fusion module is configured to: upsample the dimension of the second feature map, and convert the dimension of the upsampled second feature map to the same as the dimension of the first feature map; operate the first feature map based on the channel attention mechanism and fuse it with the first feature map to obtain a fourth feature map, operate the upsampled second feature map based on the spatial attention mechanism and fuse it with the second feature map to obtain a fifth feature map; fuse the fourth feature map and the fifth feature map to obtain a sixth feature map; calculate the global features and local features of the sixth feature map respectively, add the global features and the local features and send them into the sigmoid function to compress them to between 0 and 1 to obtain a first weight, and subtract the first weight from 1 to obtain a second weight; multiply the first weight by the fourth feature map, multiply the second weight by the fifth feature map, and accumulate the two multiplication results to obtain a third fused feature map.

[0011] According to an embodiment of the present invention, the convolution kernel size of the residual block is 3, the step size of the first convolution layer is 2, and the step size of the second convolution layer is 1.

[0012] According to an embodiment of the present invention, the first decoding module includes a decoder using an Attention-unet network, and the second decoding module includes a decoder using a multi-layer perceptron.

[0013] Another aspect of the present invention provides a fully polarized SAR image segmentation method, comprising: performing multiple residual operations on the fully polarized SAR image to obtain multiple first feature maps of different dimensions, capturing information of different sizes in the first feature map output by the last residual operation based on hole convolution and multi-scale dynamic convolution to obtain a second feature map, fusing the second feature map with the remaining first feature maps of different dimensions and then performing convolution processing to obtain a first fused feature map; wherein the first feature map output by the previous residual operation is used as the input of the next residual operation; performing multiple operations on the fully polarized SAR image based on the self-attention mechanism to obtain multiple second feature maps of different dimensions; based on the self-attention mechanism, The spatial attention mechanism and the channel attention mechanism are used to fuse multiple second feature maps of different dimensions and then perform convolution processing to obtain a second fused feature map, wherein the second feature map output by the previous operation is used as the input of the next operation; the first feature map and the corresponding second feature map are fused respectively to obtain multiple third fused feature maps, and information of different sizes in the multiple third fused feature maps is captured based on the hole convolution and the multi-scale dynamic convolution to obtain multiple third feature maps, and the multiple third feature maps are spliced ​​and convolved to obtain a fourth fused feature map; the first fused feature map, the second fused feature map, and the fourth fused feature map are weightedly fused to obtain the image segmentation result.

[0014] One aspect of the present invention provides a training method for a fully polarized SAR image segmentation network model, comprising: acquiring a SAR satellite image and a multispectral image of the same phase; processing the SAR satellite image and the multispectral image respectively and continuing to crop the overlapping area to obtain a fully polarized pseudo-color image and a visible light image; performing pixel-by-pixel data annotation based on the fully polarized pseudo-color image and the visible light image to obtain a training data set; and using the training data set to train the fully polarized SAR image segmentation network model.

[0015] The fully polarized SAR image segmentation network model, segmentation method, and training method according to the embodiments of the present invention have at least the following beneficial effects:

[0016] The full-polarimetric SAR image segmentation network model performs image processing based on Convolutional Neural Networks (CNN) and Vision Transformer (ViT) networks. It simultaneously obtains multi-scale features of CNN and ViT through a dual-branch parallel feature extraction method, and performs feature fusion through the third branch. It has both the local feature extraction capability of CNN and the global information modeling advantage of ViT, which complements each other and improves the accuracy of image segmentation.

[0017] The fully polarimetric SAR image segmentation network model sets up a MultiScale Dynamic Attention Module (MSDAM), which adaptively captures information of different sizes in the image through hole convolution and multi-scale dynamic convolution, can capture multi-scale semantic information, and improve the accuracy of image segmentation.

[0018] The fully polarimetric SAR image segmentation network model sets up a fusion module (BiAFusion module) to add a channel attention mechanism to the feature maps extracted by the CNN branch, and a spatial attention mechanism to the feature maps extracted by the ViT branch. Subsequently, adaptive weights are assigned to the two feature maps by adding the feature maps, thereby achieving effective fusion of the feature maps obtained by CNN and ViT at the same resolution, thereby improving the accuracy of image segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The above and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0020] Figure 1 A schematic diagram of a structure of a fully polarized SAR image segmentation network according to an embodiment of the present invention is shown;

[0021] Figure 2 Schematically shows a structural block diagram of a multi-scale dynamic attention module according to an embodiment of the present invention;

[0022] Figure 3 A structural block diagram of a fusion module according to an embodiment of the present invention is schematically shown;

[0023] Figure 4 A schematic diagram of a decoding module according to an embodiment of the present invention is shown;

[0024] Figure 5 A flowchart of a fully polarized SAR image segmentation method according to an embodiment of the present invention is schematically shown;

[0025] Figure 6 A flowchart of a training method for a full polarization SAR image segmentation network model according to an embodiment of the present invention is schematically shown;

[0026] Figure 7 A schematic diagram of a selected data area in a process of preparing a training data set according to an embodiment of the present invention is shown;

[0027] Figure 8 A diagram schematically shows a result of cropping overlapping regions of GF3 image and Landsat image data of the same phase according to an embodiment of the present invention;

[0028] Fig. 9 A diagram schematically shows a data annotation example according to an embodiment of the present invention;

[0029] Fig.10 A block diagram of an electronic device suitable for implementing the method described above according to an embodiment of the present invention is schematically shown. DETAILED DESCRIPTION

[0030] Below, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of concepts of the present invention.

[0031] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0032] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0033] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0034] In the process of realizing the present invention, the applicant found that the boundaries between different categories of sea ice in SAR sea ice images are very obvious, less affected by speckle noise, and suitable for segmentation using semantic segmentation methods that make full use of contextual information. CNN and ViT are two mainstream feature extraction modules. CNN uses local receptive fields when processing images and is good at capturing local features of images, but it will cause the loss of spatial information during convolution and pooling, and is insensitive to position. ViT introduces the attention mechanism of the Transform model, which can model the global information of the entire image and better understand the structure and semantics of the image, but is insensitive to local features.

[0035] In view of this, an embodiment of the present invention provides a fully polarized SAR image segmentation network (Sea ice-Transformer fusion based network, SI-CTFNet) based on CNN and VIT. Aiming at the difficulty of interpreting SAR sea ice images, the image segmentation network reduces the downsampling ratio in the dual-branch feature extraction framework to retain more original image information, and modifies the Transformer branch to output a multi-scale feature map like the CNN branch. In view of the different proportions of sea ice of different categories in SAR sea ice images, the image segmentation network sets an MSDAM module to adaptively capture information of different sizes in the image through hole convolution and multi-scale dynamic convolution. The image segmentation network designs a BiAFusion fusion module, which adds a channel attention mechanism and a spatial attention mechanism to the feature maps of CNN and ViT respectively, and subsequently connects an adaptive parameter module to achieve effective fusion of feature maps obtained by CNN and ViT at the same resolution. The following is a detailed introduction in conjunction with specific embodiments.

[0036] The fully polarized SAR image segmentation network provided by the embodiment of the present invention may include: a first branch, a second branch, a third branch and a weighted fusion module, wherein the first branch, the second branch and the third branch are connected in parallel and then connected to the weighted fusion module.

[0037] The first branch is configured to perform multiple residual operations on the fully polarized SAR image to obtain multiple first feature maps of different dimensions, capture information of different sizes in the first feature map output by the last residual operation based on void convolution and multi-scale dynamic convolution, and obtain a second feature map, fuse the second feature map with the other first feature maps of different dimensions and then perform convolution processing to obtain a first fused feature map; wherein the first feature map output by the previous residual operation is used as the input of the next residual operation.

[0038] The second branch is configured to perform multiple operations on the fully polarized SAR image based on the self-attention mechanism to obtain multiple second feature maps of different dimensions, and to fuse the multiple second feature maps of different dimensions based on the spatial attention mechanism and the channel attention mechanism and then perform convolution processing to obtain a second fused feature map, wherein the second feature map output by the previous operation is used as the input of the next operation.

[0039] The third branch is configured to fuse the first feature map and the corresponding second feature map respectively to obtain multiple third fused feature maps, capture information of different sizes in the multiple third fused feature maps based on void convolution and multi-scale dynamic convolution to obtain multiple third feature maps, and splice and convolve the multiple third feature maps to obtain a fourth fused feature map.

[0040] The weighted fusion module is configured to perform weighted fusion on the first fusion feature map, the second fusion feature map, and the fourth fusion feature map to obtain an image segmentation result.

[0041] Figure 1 The structural block diagram of the full polarimetric SAR image segmentation network according to an embodiment of the present invention is schematically shown.

[0042] like Figure 1 As shown, in a possible implementation of an embodiment of the present invention, the first branch can be improved by deeplabV3+, the lower branch can be improved by segformer, and the middle is a dual-branch fusion module, and the feature maps of the corresponding layers of CNN and VIT are input into the fusion network to obtain the fused feature maps. Three learnable weight parameters α, β, and γ are set at the output end. After the three branches independently complete the segmentation of the input image, they are multiplied and added with the three weights, and finally the segmentation result is obtained through a convolution block. The above process can be expressed by the following formula:

[0043]

[0044] Among them, F out is the final output prediction result, F cnn 、F fuse 、F trans They represent the prediction results of the three branches respectively, and Conv represents the convolution operation.

[0045] Continue reading Figure 1 On the basis of the above embodiment, the first branch includes at least two residual blocks connected in series, a multi-scale dynamic attention module, a first decoding module and a convolution layer; the at least two residual blocks connected in series are configured to perform multiple residual operations on the fully polarized SAR image to obtain multiple first feature maps of different dimensions, wherein the output of the previous residual operation module is used as the input of the next residual operation module, the multi-scale dynamic attention module is configured to capture information of different sizes in the first feature map output by the last residual operation based on void convolution and multi-scale dynamic convolution to obtain a second feature map, and the first decoding module and the convolution layer are configured to fuse the second feature map with the other first feature maps of different dimensions and then perform convolution processing to obtain a first fused feature map.

[0046] For example, the input of the first branch can be an H×W×3 image, and the backbone network is a CNN, such as a modified Resnet18. Resnet18 can be composed of four residual blocks. The structure of the residual block is as follows: Figure 1As shown in the dotted box on the upper left, the convolution kernel size can be 3, the stride of the first convolution layer can be 2, and the stride of the second convolution layer can be 1. Therefore, the input full polarimetric SAR image can be sequentially passed through the convolution layer (e.g., 7×7 convolution layer), batch normalization layer (Batch Normalization, BN), activation layer (Relu activation function) and then input into the residual block. After each feature map is operated by a residual block, the dimension is reduced to half of the original, that is, the dimension of the feature map output by each residual block is H / 2×W / 2, H / 4×W / 4, H / 8×W / 8, H / 16×W / 16 respectively. The feature map output by the last residual block is input into the MultiScale Dynamic Attention Module (MSDAM) module, which is mainly improved on the Atrous Spatial Pyramid Pooling (ASPP) module in the original deeplabV3+, adding a channel attention mechanism and a multi-scale feature fusion module. The feature map output by MSDAM can be used as the input of the decoder together with the shallow features output by the other three residual blocks. The first decoding module can directly use the decoder of the Attention-unet network, which will be described in detail later. The output of the decoder can be adjusted to 32 channels through a 1×1 convolution layer, and multiplied by the learnable parameter α to obtain the output of this branch.

[0047] Continue reading Figure 1 On the basis of the above embodiment, the second branch includes a number of self-attention operation modules, a second decoding module and a convolution layer which are connected in series in the same number as the number of residual modules; at least two residual modules connected in series in the same number are configured to perform multiple operations on the fully polarized SAR image to obtain multiple second feature maps of different dimensions, and the second decoding module and the convolution layer are configured to fuse the multiple second feature maps of different dimensions based on the spatial attention mechanism and the channel attention mechanism and then perform convolution processing to obtain a second fused feature map. The output of the second decoding module can adjust the number of channels to 32 through a 1×1 convolution layer, and multiply it by the learnable parameter γ to obtain the output of the branch.

[0048] For example, the second branch can use a segformer network, with an input of an H×W×3 image, which first passes through the image encoding layer (Overlap patch Embedings) and then enters the self-attention operation module. The Segformer backbone network consists of cascaded self-attention operation modules (transformer blocks), and the input of each transformer block is a feature map after block and position encoding. Then, a multilayer perceptron (MLP) is used as the second decoding module to fuse feature maps of multiple scales to obtain the prediction result. The transformer block can include a self-attention mechanism (Efficient Self=Attention), a feedforward neural network (Mix-FFN), and block fusion (Overlap patch Merging) connected in sequence.

[0049] Continue reading Figure 1 On the basis of the above embodiment, the third branch includes fusion modules (F), multi-scale dynamic attention modules, a first decoding module and a convolution layer, the number of which is equal to the number of residual modules; each fusion module is configured to fuse a first feature map and a corresponding second feature map; the multi-scale dynamic attention module is configured to capture information of different sizes in multiple third fused feature maps based on hole convolution and multi-scale dynamic convolution, and the first decoding module and the convolution layer splice and convolve the output of the multi-scale dynamic attention module to obtain a third fused feature map.

[0050] For example, the third branch can be composed of four fusion modules (BiAFF modules), which receive the feature maps output by the corresponding modules in the CNN and segformer backbone networks respectively, and complete the fusion of the two types of feature maps. The four BiAFF modules are connected to the MSDAM module to further extract deep features, and the results are simultaneously input into the decoding module. So far, the subsequent structure of this branch is the same as that of the upper branch, and the output result is multiplied by the learnable parameter β to obtain the output of this branch. The structure of the BiAFF module will be described in detail later.

[0051] Figure 2 The structural block diagram of a multi-scale dynamic attention module according to an embodiment of the present invention is schematically shown.

[0052] like Figure 2 As shown, based on the above embodiment, the MSDAM module may include: a dilated convolution pooling unit (ASPP), a multi-scale extraction unit (MultiScale), a splicing unit, a convolution unit and a channel attention unit;

[0053] The atrous convolution pooling unit is configured to perform processing at different dilation rates, convolution processing, and pooling processing on the first feature map or the third fused feature map to extract and fuse information of different sizes.

[0054] The multi-scale extraction unit is configured to extract and fuse information of features at multiple scales and in different receptive fields from the first feature map or the third fused feature map.

[0055] The splicing unit is configured to splice the first feature map or the third fused feature map with the corresponding output of the dilated convolution pooling unit and the multi-scale extraction unit to obtain a spliced ​​feature map.

[0056] The convolution unit is configured to perform convolution on the concatenated feature map to reduce the channel dimension.

[0057] The channel attention unit is configured to obtain the weight corresponding to each channel through global pooling and fully connected layers, and then multiply it channel by channel with the concatenated feature map after dimensionality reduction to obtain the second feature map corresponding to the first feature map or the third feature map corresponding to the third fused feature map.

[0058] For example, in the original deeplabV3+ network, this position is the ASPP unit, which expands the receptive field without losing resolution by setting different dilation rates, capturing more contextual semantic information. However, too large a dilation rate will cause small targets to lose information, resulting in grid effect and boundary effect. Therefore, based on the original ASPP module, the present invention adds a multi-scale module, uses dynamic multi-scale filters to subdivide semantics, and captures multi-scale semantic information.

[0059] The input of the MSDAM module can be a feature map of H×W×C1, which contains two branches. One branch is the ASPP unit in the original deeplabV3+. In order to reduce the problem of information loss, the original dilation rate is reduced to {3,6,9}, and the other structures are the same as the original ASPP unit. The other branch is the multi-scale extraction unit (Multiscale). This unit can be composed of four DCM subunits with variable k values.

[0060] Each scale extraction unit in the multi-scale extraction unit includes two extraction branches, one extraction branch is configured to reduce the dimension of the first feature map or the third fused feature map, and the other extraction branch is configured to perform pooling and dimensionality reduction on the first feature map or the third fused feature map, and then the outputs of the two extraction branches are fused based on depthwise separable convolution.

[0061] For example, each DCM subunit is further divided into two branches. One branch reduces the feature map dimension to H×W×C2 through 1×1 convolution, and the other branch converts the feature map to k×k×C2 through adaptive pooling and 1×1 convolution. Finally, the two are fused through a depth-wise conv operation. The output of the ASPP unit and the MultiScale unit is concatenated with the original input feature map to obtain a feature map of H×W×(C2×3) dimensions, and then the channel dimension is reduced through a 1×1 convolution unit. The MSDAM module ends with a channel attention unit, which obtains the weight corresponding to each channel through global pooling and a fully connected layer, and then multiplies it channel by channel with the original feature map to obtain the final output feature map.

[0062] Figure 3 The structural block diagram of the fusion module according to an embodiment of the present invention is schematically shown.

[0063] like Figure 3 As shown, based on the above embodiment, the dimension of the first feature map is twice the dimension of the second feature map; the fusion module is configured to: upsample the dimension of the second feature map, and convert the dimension of the upsampled second feature map to the same as the dimension of the first feature map; based on the channel attention mechanism, the first feature map is operated and fused with the first feature map to obtain a fourth feature map, based on the spatial attention mechanism, the upsampled second feature map is operated and fused with the second feature map to obtain a fifth feature map; the fourth feature map and the fifth feature map are fused to obtain a sixth feature map; the global features and local features of the sixth feature map are calculated respectively, the global features and the local features are added and sent to the sigmoid function to be compressed between 0 and 1 to obtain a first weight, and the first weight is subtracted from 1 to obtain a second weight; the first weight is multiplied by the fourth feature map, the second weight is multiplied by the fifth feature map, and the two multiplication results are accumulated to obtain a third fused feature map.

[0064] Since CNN is good at capturing local information of images, while transformer is good at modeling the global information of the entire image and better understanding the structure and semantics of the image, the present invention designs a BiAFusion module as a fusion module to realize the fusion operation of feature maps extracted by CNN and transformer.

[0065] Since the dimension of the first feature map (fc) extracted by CNN is twice that of the second feature map (ft) extracted by transformer, ft needs to undergo an upsampling operation first to keep the dimensions of the two feature maps consistent. Since fc focuses more on local information and ft focuses more on global information, it is decided to add a channel attention mechanism to fc and a spatial attention mechanism to ft in the selection of the attention mechanism, and add the two to obtain a feature map with both local features and global semantic information. So far, although the fusion of the two types of feature maps has been completed, the importance of the two types of feature maps may not be exactly the same. Therefore, the present invention can also add a trainable weight calculation module. The module takes the fused feature map as input, divides two branches into two branches to respectively calculate the global features and local features of the current feature map, and then adds the two branches to the sigmoid function to compress the result to between 0 and 1. Multiply this weight with fc with the channel attention mechanism added, subtract the weight from 1, and then multiply it with ft with the spatial attention mechanism added, and finally add the two product results to obtain the third fused feature map. This process can be described by the following formula:

[0066]

[0067] Among them, CA represents channel attention operation, SA represents spatial attention operation, local and global represent the acquisition of local features and global features respectively, and fc and ft are the input feature maps respectively.

[0068] It can be understood that the above-mentioned number limitation on the modules or units is not limited to four, and can be other numbers, which are selected according to actual application requirements and are not limited by the present invention.

[0069] Since shallow features have rich spatial detail information, such as color, texture and edge, but lack deep semantic information, and deep features have rich semantic information but low spatial resolution, organically combining feature maps of different scales can often effectively improve segmentation accuracy. The present invention can input feature maps of different scales into the decoder at the same time.

[0070] Figure 4 The structural block diagram of a decoding module according to an embodiment of the present invention is schematically shown.

[0071] like Figure 4 As shown, Figure 4 The upper part is a structural block diagram of the first decoding module, and the lower part is a structural block diagram of the second decoding module.

[0072] The specific implementation of the first decoding module is:

[0073] Assume that the deep feature map fd and the shallow feature map fs are input, and fd is first upsampled to the same dimension as fs, and then input into the attention gate to obtain the fused feature map. Then the fused feature map is concatenated with the deep feature map, and the channel dimension is adjusted through the 1x1 convolution module, and the above process is repeated with the next shallow feature. The above process can be expressed as follows:

[0074]

[0075] Among them, f d and f s They represent the deep feature map and shallow feature map of the input respectively, upsample represents the upsampling operation, cat represents the concatenation by channel dimension, and AG represents the attention gate operation.

[0076] The specific implementation of the second decoding module is:

[0077] Feature maps of different scales are concatenated together through upsampling and MLP, and finally an MLP layer is connected to obtain the output result.

[0078] Based on the above image segmentation network model, an embodiment of the present invention further provides a full-polarization SAR image segmentation method.

[0079] Figure 5 The flowchart of the full polarimetric SAR image segmentation method according to an embodiment of the present invention is schematically shown.

[0080] like Figure 5 As shown, the full polarization SAR image segmentation method may include S510 to S540.

[0081] In S510, multiple residual operations are performed on the fully polarized SAR image to obtain multiple first feature maps of different dimensions. Based on the dilated convolution and the multi-scale dynamic convolution, the information of different sizes in the first feature map output by the last residual operation is captured to obtain the second feature map. The second feature map is fused with the other first feature maps of different dimensions and then convolved to obtain the first fused feature map.

[0082] Among them, the first feature map output by the previous residual operation is used as the input of the next residual operation.

[0083] In S520, the fully polarized SAR image is operated multiple times based on the self-attention mechanism to obtain multiple second feature maps of different dimensions, and the multiple second feature maps of different dimensions are fused based on the spatial attention mechanism and the channel attention mechanism and then convolution processing is performed to obtain a second fused feature map.

[0084] Among them, the second feature map output by the previous operation is used as the input of the next operation.

[0085] In S530, the first feature map and the corresponding second feature map are fused respectively to obtain multiple third fused feature maps, and information of different sizes in the multiple third fused feature maps is captured based on the void convolution and the multi-scale dynamic convolution to obtain multiple third feature maps. The multiple third feature maps are spliced ​​and convolved to obtain a fourth fused feature map.

[0086] In S540, the first fused feature map, the second fused feature map, and the fourth fused feature map are weightedly fused to obtain an image segmentation result.

[0087] It should be noted that the specific implementation details of the polarimetric SAR image segmentation method and the technical effects brought about are similar to or identical to the specific implementation details of the polarimetric SAR image segmentation network model and the technical effects brought about. For any details not included, please refer to the method embodiment part, which will not be repeated here.

[0088] Based on the above embodiments, the present invention also provides a training method for a full-polarization SAR image segmentation network model.

[0089] Figure 6 The flowchart of the training method of the full polarization SAR image segmentation network model according to an embodiment of the present invention is schematically shown.

[0090] like Figure 6 As shown, the training method flow may include S610 to S640.

[0091] At S610 , a SAR satellite image and a multispectral image of the same phase are acquired.

[0092] In S620, the SAR satellite image and the multispectral image are processed respectively and the overlapping areas are clipped to obtain a full polarization pseudo-color image and a visible light image.

[0093] At S630, pixel-by-pixel data annotation is performed based on the full polarization pseudo-color image and the visible light image to obtain a training data set.

[0094] At S640, a training data set is used to train a full polarimetric SAR image segmentation network model.

[0095] Figure 7 The diagram schematically shows a data region selected during the training data set preparation process according to an embodiment of the present invention. Figure 8 The figure schematically shows the cropping result of the overlapping area of ​​the GF3 image and the Landsat image data of the same phase according to an embodiment of the present invention. Fig. 9 A diagram schematically shows a data annotation instance according to an embodiment of the present invention.

[0096] like Figure 7As shown in the figure, the small squares can represent GF3 full polarization strip pattern data, and the large squares can represent Landsat multispectral data. The width of the GF3 satellite SAR sea ice image can be 30km, and the nominal resolution can be 8m. The resolution of the Landsat-8 satellite multispectral sea ice image can be 30m and 15m.

[0097] There are a lot of overlaps between the visible light sea ice image and the SAR sea ice image. The GDAL library can be used to crop the overlapping areas. First, input the two images to be cropped and obtain the longitude and latitude of their four corner points. The longitude and latitude of the four corner points of the quadrilateral in the overlapping area are determined and calculated using the two sets of longitude and latitude. The cropping function provided in the GDAL library is used to crop the two images using the longitude and latitude of the four corner points, and the two images are resampled to the same spatial resolution at the same time. The cropping result is shown below. Figure 8 As shown, the left image may be a GF3 full polarization pseudo-color image, and the right image may be a Landsat visible light image.

[0098] Different classification standards for sea ice are defined based on the sea ice conditions in the North and South Poles and high latitudes, among which the classification standard based on sea ice thickness is widely used. Fig. 9 As shown, the present invention refers to the Landsat8 optical sea ice image and the sea ice classification standard formulated by WMO, and divides the sea ice into 4 categories in the selected data, namely open water (open water / OW), new ice (new ice / NI), young ice (Young ice / YI) and first year ice (first year ice / FYI). It can be seen from the Landsat8 satellite optical image that seawater and new ice are mostly darker areas, among which seawater is purer. Young ice can be divided into gray ice and gray-white ice, which can be well distinguished from the optical image. One-year ice is also called white ice, and the white part in the optical image is one-year ice. Although there are many types of sea ice in the HH polarized SAR image that cannot be distinguished only by backscattering intensity, since the reflection characteristics of different sea ice to electromagnetic waves of different polarization combinations are also different, they can be distinguished in pseudo-color images. The characteristics of OW and NI are similar to those in optical images. Due to the smooth surface, the backscattering coefficient is small, and they are usually darker areas in SAR images. The scattering intensity of YI is the largest among the four types of sea ice, and the structure of FYI is more complex, often containing multiple scattering types.

[0099] In order to verify the feasibility and advantages of the full-polarization SAR image segmentation network model, segmentation method and training method provided by the present invention, the present invention also provides some experimental results.

[0100] The embodiments of the present invention can use five indicators to evaluate the classification effect of the model. PA (Pixel Accuracy), CPA (Class Pixel Accuracy), mPA (mean Pixel Accuracy), IoU (Intersection over Union) and mIoU (mean Intersection over Union). PA represents the ratio of the number of correctly classified pixels in all categories to all pixels, CPA represents the ratio of the number of correctly classified pixels in a single category to all pixels, and mPA is obtained by averaging CPA. IoU represents the overlap rate of the predicted result area and the true value area in a single category, that is, the ratio of their intersection to union, and mIoU is obtained by averaging IoU. The calculation formula is as follows:

[0101]

[0102] The meanings of TP, FP, TN, and FN can be found in Table 1, and N is the number of categories.

[0103] Table 1

[0104]

[0105] The test method can be:

[0106] 1) The dataset preparation method proposed in the present invention can be used to select image samples of size 512×512 from the selected 12 GF3 sea ice images, and annotate them with reference to the Landsat images at the same location. A total of 600 images were selected and annotated, of which 500 were used for training and 100 for testing.

[0107] 2) Divide the obtained dataset into training set and test set, use the training set to train our model, and verify it on the test set.

[0108] 3) Compare with other classification methods to verify the improvement of SI-CTFNet in classification accuracy.

[0109] The test results can be shown in Table 2:

[0110] Table 2

[0111]

[0112] Fig.10 A block diagram of an electronic device suitable for implementing the method described above according to an embodiment of the present invention is schematically shown. Fig.10 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0113] like Fig.10 As shown, the electronic device 1000 according to an embodiment of the present invention includes a processor 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage part 1008 to a random access memory (RAM) 1003. The processor 1001 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 1001 may also include an onboard memory configured for cache purposes. The processor 1001 may include a single processing unit or multiple processing units configured to perform different actions of the method flow according to an embodiment of the present invention.

[0114] In RAM 1003, various programs and data required for the operation of electronic device 1000 are stored. Processor 1001, ROM 1002 and RAM 1003 are connected to each other via bus 1004. Processor 1001 performs various operations of the method flow according to the embodiment of the present invention by executing the program in ROM 1002 and / or RAM 1003. It should be noted that the program can also be stored in one or more memories other than ROM 1002 and RAM 1003. Processor 1001 can also perform various operations of the method flow according to the embodiment of the present invention by executing the program stored in the one or more memories.

[0115] According to an embodiment of the present invention, the electronic device 1000 may further include an input / output (I / O) interface 1005, which is also connected to the bus 1004. The electronic device 1000 may further include one or more of the following components connected to the input / output (I / O) interface 1005: an input portion 1006 including a keyboard, a mouse, etc.; an output portion 1007 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 1008 including a hard disk, etc.; and a communication portion 1009 including a network interface card such as a LAN card, a modem, etc. The communication portion 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the input / output (I / O) interface 1005 as needed. A removable medium 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1010 as needed, so that a computer program read therefrom is installed into the storage portion 1008 as needed.

[0116] According to an embodiment of the present invention, the method flow according to an embodiment of the present invention can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program contains a program code configured to execute the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1009, and / or installed from the removable medium 1011. When the computer program is executed by the processor 1001, the above-mentioned functions defined in the system of the embodiment of the present invention are executed. According to an embodiment of the present invention, the system, equipment, device, module, unit, etc. described above can be implemented by a computer program module.

[0117] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiment; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present invention is implemented.

[0118] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium. For example, it may include, but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, apparatus, or device.

[0119] For example, according to an embodiment of the present invention, the computer-readable storage medium may include the ROM 1002 and / or the RAM 1003 described above and / or one or more memories other than the ROM 1002 and the RAM 1003 .

[0120] An embodiment of the present invention also includes a computer program product, which includes a computer program, which contains program code configured to execute the method provided by the embodiment of the present invention. When the computer program product runs on an electronic device, the program code is configured to enable the electronic device to implement the training method and prediction method provided by the embodiment of the present invention.

[0121] When the computer program is executed by the processor 1001, the above functions defined in the system / device of the embodiment of the present invention are executed. According to the embodiment of the present invention, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0122] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 1009, and / or installed from the removable medium 1011. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0123] According to an embodiment of the present invention, the program code configured to execute the computer program provided by the embodiment of the present invention can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0124] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram may represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions configured to implement the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions. It will be understood by those skilled in the art that the features recorded in the various embodiments of the present invention can be combined and / or combined in various ways, even if such a combination or combination is not explicitly recorded in the present invention. In particular, without departing from the spirit and teachings of the present invention, the features recorded in the various embodiments of the present invention can be combined and / or combined in various ways. All these combinations and / or combinations fall within the scope of the present invention.

[0125] The embodiments of the present invention are described above. However, these embodiments are only for the purpose of illustration, and are not intended to limit the scope of the present invention. Although each embodiment is described above, it does not mean that the measures in each embodiment cannot be used in combination advantageously. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A fully polarimetric SAR image segmentation network system, It is characterized in that include: The first branch, the second branch and the third branch are connected in parallel, and a weighted fusion module; The first branch is configured to perform multiple residual operations on the fully polarized SAR image to obtain multiple first feature maps of different dimensions, capture information of different sizes in the first feature map output by the last residual operation based on the dilated convolution and the multi-scale dynamic convolution, obtain the second feature map, fuse the second feature map with the other first feature maps of different dimensions and then perform convolution processing to obtain the first fused feature map; wherein the first feature map output by the previous residual operation is used as the input of the next residual operation; The second branch is configured to perform multiple operations on the fully polarized SAR image based on a self-attention mechanism to obtain multiple second feature maps of different dimensions, and to fuse the multiple second feature maps of different dimensions based on a spatial attention mechanism and a channel attention mechanism and then perform convolution processing to obtain a second fused feature map, wherein the second feature map outputted by the previous operation is used as an input of the next operation; The third branch is configured to fuse the first feature map and the corresponding second feature map respectively to obtain multiple third fused feature maps, capture information of different sizes in the multiple third fused feature maps based on hole convolution and multi-scale dynamic convolution to obtain multiple third feature maps, and perform splicing and convolution processing on the multiple third feature maps to obtain a fourth fused feature map; The weighted fusion module is configured to perform weighted fusion on the first fusion feature map, the second fusion feature map, and the fourth fusion feature map to obtain an image segmentation result; Wherein, the first branch includes at least two residual blocks connected in series, a multi-scale dynamic attention module, a first decoding module and a convolution layer; the at least two residual blocks connected in series are configured to perform multiple residual operations on the fully polarized SAR image to obtain multiple first feature maps of different dimensions, wherein the output of the previous residual operation module is used as the input of the next residual operation module, the multi-scale dynamic attention module is configured to capture information of different sizes in the first feature map output by the last residual operation based on the hole convolution and the multi-scale dynamic convolution to obtain the second feature map, the first decoding module and the convolution layer are configured to fuse the second feature map with the other first feature maps of different dimensions and then perform convolution processing to obtain the first fused feature map; The second branch includes a number of self-attention operation modules, a second decoding module, and a convolution layer that are connected in series in sequence, which is equal to the number of the residual blocks; at least two residual blocks connected in series in sequence are configured to perform multiple operations on the fully polarized SAR image to obtain multiple second feature maps of different dimensions, and the second decoding module and the convolution layer are configured to fuse the multiple second feature maps of different dimensions based on a spatial attention mechanism and a channel attention mechanism and then perform convolution processing to obtain a second fused feature map; The third branch includes fusion modules, multi-scale dynamic attention modules, first decoding modules and convolution layers, the number of which is equal to the number of the residual blocks; each fusion module is configured to fuse a pair of the first feature maps and a corresponding pair of the second feature maps; the multi-scale dynamic attention module is configured to capture information of different sizes in the plurality of third fused feature maps based on hole convolution and multi-scale dynamic convolution, and the first decoding module and the convolution layer perform splicing and convolution processing on the output of the multi-scale dynamic attention module to obtain a third fused feature map; The multi-scale dynamic attention module includes: a hole convolution pooling unit, a multi-scale extraction unit, a splicing unit, a convolution unit and a channel attention unit; The atrous convolution pooling unit is configured to perform processing with different dilation rates, convolution processing and pooling processing on the first feature map or the third fused feature map to extract and fuse information of different sizes; The multi-scale extraction unit is configured to extract and fuse information of features at multiple scales and in different receptive fields from the first feature map or the third fused feature map; The splicing unit is configured to splice the first feature map or the third fused feature map with the corresponding output of the dilated convolution pooling unit and the multi-scale extraction unit to obtain a spliced ​​feature map; The convolution unit is configured to perform convolution processing on the concatenated feature map to reduce the channel dimension; The channel attention unit is configured to obtain the weight corresponding to each channel through global pooling and a fully connected layer, and then multiply it channel by channel with the concatenated feature map after dimensionality reduction to obtain the second feature map corresponding to the first feature map or the third feature map corresponding to the third fused feature map.

2. The segmented network system according to claim 1, It is characterized in that Each scale extraction unit in the multi-scale extraction unit includes two extraction branches, one extraction branch is configured to reduce the dimension of the first feature map or the third fused feature map, and the other extraction branch is configured to perform pooling and dimensionality reduction processing on the first feature map or the third fused feature map, and then the outputs of the two extraction branches are fused based on depth-separable convolution.

3. The segmented network system according to claim 1, It is characterized in that The different expansion rates include 3, 6, and 9.

4. The segmented network system according to claim 1, It is characterized in that The dimension of the first feature map is twice the dimension of the second feature map; and the fusion module is configured as follows: Upsampling the dimension of the second feature map, and converting the dimension of the upsampled second feature map to be the same as the dimension of the first feature map; The first feature map is operated based on the channel attention mechanism and merged with the first feature map to obtain a fourth feature map, and the upsampled second feature map is operated based on the spatial attention mechanism and merged with the second feature map to obtain a fifth feature map; fusing the fourth feature map and the fifth feature map to obtain a sixth feature map; Respectively calculating the global features and the local features of the sixth feature map, adding the global features and the local features and sending them into a sigmoid function to be compressed to between 0 and 1 to obtain a first weight, and subtracting the first weight from 1 to obtain a second weight; The first weight is multiplied by the fourth feature map, the second weight is multiplied by the fifth feature map, and the two multiplication results are accumulated to obtain the third fused feature map.

5. The segmented network system according to claim 1, It is characterized in that The convolution kernel size of the residual block is 3, the stride of the first convolution layer is 2, and the stride of the second convolution layer is 1.

6. The segmented network system according to claim 1, It is characterized in that The first decoding module includes a decoder using an Attention-unet network, and the second decoding module includes a decoder using a multi-layer perceptron.

7. A method for segmenting a fully polarized SAR image based on the segmentation network system according to any one of claims 1 to 6, It is characterized in that include: Performing multiple residual operations on the fully polarized SAR image to obtain multiple first feature maps of different dimensions, capturing information of different sizes in the first feature map output by the last residual operation based on dilated convolution and multi-scale dynamic convolution to obtain a second feature map, fusing the second feature map with the remaining first feature maps of different dimensions and then performing convolution processing to obtain a first fused feature map; wherein the first feature map output by the previous residual operation is used as the input of the next residual operation; Based on the self-attention mechanism, the fully polarized SAR image is operated multiple times to obtain multiple second feature maps of different dimensions, and based on the spatial attention mechanism and the channel attention mechanism, the multiple second feature maps of different dimensions are fused and then convolved to obtain a second fused feature map, wherein the second feature map outputted from the previous operation is used as the input of the next operation; The first feature map and the corresponding second feature map are respectively fused to obtain multiple third fused feature maps, information of different sizes in the multiple third fused feature maps are captured based on hole convolution and multi-scale dynamic convolution to obtain multiple third feature maps, and the multiple third feature maps are spliced ​​and convolved to obtain a fourth fused feature map; The first fused feature map, the second fused feature map, and the fourth fused feature map are weightedly fused to obtain an image segmentation result.

8. A training method for the full polarimetric SAR image segmentation network system according to any one of claims 1 to 6, It is characterized in that include: Acquire simultaneous SAR satellite images and multispectral images; The SAR satellite image and the multispectral image are processed respectively and the overlapping areas are cut to obtain a full polarization pseudo-color image and a visible light image; Performing pixel-by-pixel data annotation based on the fully polarized pseudo-color image and the visible light image to obtain a training data set; The training data set is used to train the fully polarimetric SAR image segmentation network system.