SAR image weak supervision semantic segmentation method based on decision optimization pseudo label generation

Abstract

The invention discloses an SAR image weak supervision semantic segmentation method based on decision optimization pseudo label generation, and the method comprises the following steps: 1, training a multi-label classification network, and obtaining class activation mapping results of different layers of a neural network and pixel-level prediction pseudo labels; 2, fusing different levels of class activation mapping results to obtain a class activation mapping result with higher confidence; step 3, forming a decision by utilizing the relation between different layers of pseudo labels on the pixels, and extracting multi-layer decision pixel relation information; 4, constructing a relation network among pixels, and obtaining a centroid displacement vector and boundary features of each pixel in the image; 5, training a relation network between pixels by using the extracted pixel relation information to obtain a pixel semantic affinity matrix; step 6, performing pseudo tag optimization by using the pixel semantic affinity matrix; and step 7, training a semantic segmentation network by using the optimized pixel-level pseudo labels, and completing SAR image weak supervision semantic segmentation. According to the method provided by the invention, the high-quality polarized SAR image pixel-level pseudo tag can be generated, and pixel-level semantic segmentation of the SAR image under the condition of image-level annotation weak supervision is realized.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of SAR image processing, and particularly relates to a SAR image weak supervision semantic segmentation method based on decision optimization pseudo label generation. BACKGROUND

[0002] Polarimetric synthetic aperture radar can observe targets under all-weather conditions and effectively obtain rich feature information of ground targets. Therefore, polarimetric SAR images have been widely used in forest monitoring, urban planning and ocean observation and the like.

[0003] Polarimetric SAR image classification, as one of the important contents in polarimetric SAR image interpretation, has attracted extensive attention. With the rapid development of deep learning technology, polarimetric SAR image classification methods based on deep learning have also emerged in an endless stream. Deep learning usually needs a large amount of data labels to provide support, however, pixel-level labeling of polarimetric SAR images needs to consume a large amount of human and material resources, resulting in limitations of polarimetric SAR data sets in data labeling. The weak supervision learning method taking image-level labels as supervision information can effectively reduce the labeling cost of polarimetric SAR data sets and alleviate the problem of insufficient pixel-level label labeling. However, due to the characteristics of complex target scattering characteristics and wide spatial distribution range of polarimetric SAR images, it is more difficult to obtain high-quality pixel-level pseudo labels using image-level labels. SUMMARY

[0004] The problem to be solved by the application is that pixel-level labeling of polarimetric SAR images is difficult to obtain under the background of deep learning, and the application provides a SAR image weak supervision semantic segmentation method based on decision optimization pseudo label generation, which has simple structure and reasonable design, uses image-level labels of SAR images to supervise a multi-label classification network, obtains class activation mapping of different levels of the network, and is used for fusion to generate pixel-level pseudo labels with high confidence; the pseudo labels obtained by the network at different levels form multi-layer decisions on the relationship between pixels, improve the reliability of the extracted relationship between pixels, improve the optimization result of the pseudo labels, and realize semantic segmentation of polarimetric SAR images under the condition of image-level weak supervision.

[0005] To solve the above technical problems, the technical scheme adopted by the application is that the SAR image weak supervision semantic segmentation method based on decision optimization pseudo label generation comprises the following steps:

[0006] Step one, training a multi-label classification network to obtain class activation mapping results of different layers of the neural network and pixel-level prediction pseudo labels;

[0007] Step 101, inputting a polarimetric SAR image and a corresponding image-level label into a multi-label classification network, and using a multi-label classification loss lossbce The multi-label classification network uses ResNet-50 as a backbone network;

[0008] Step 102, from the trained multi-label classification network, using the feature maps of the last four stages of the network and the backpropagation gradient, obtaining class activation mapping results of the shallow and deep layers of the network Wherein, A1, A2, A3 represent the class activation mapping results of the shallow layers of the network, A4 represents the class activation mapping results of the deep layers of the network, H and W represent the length and width of the polarized SAR image respectively, and C represents the number of classes, which is also the channel dimension of the class activation mapping results.

[0009] Step 103, for the class activation mapping results obtained by different layers of the network Each position returns the maximum value in the channel dimension C, and the channel index corresponding to the maximum value is taken as the predicted class, and then the pixel-level prediction pseudo-labels of different layers of the network are obtained Wherein, represents the prediction pseudo-label obtained from the class activation mapping A i

[0010] Step two, fuse the class activation mapping results of different layers to obtain class activation mapping results with higher confidence;

[0011] Step 201, fuse the class activation mapping results A1, A2, A3, A4 obtained by different layers of the network to generate high-confidence fusion class activation mapping A fuse

[0012] Step three, use the pseudo-labels of different layers of the network to make decisions on the relationship between pixels, and extract multi-layer decision pixel relationship information;

[0013] Step 301, according to the class activation mapping results A1, A2, A3, A4, the pseudo-labels obtained by different layers of the network Set the high-confidence class region and the low-confidence region, and sort the class activation mapping values corresponding to each class in the prediction pseudo-label Set the value at the 30th position as the threshold, and set the pixel positions greater than the threshold as the high-confidence class region, and the remaining pixel positions as the low-confidence region.

[0014] Step 302, construct a set of positive and negative sample pixel pairs P i and N i , in the pseudo-label after setting the high-confidence class region and the low-confidence region, set the center pixel position x c , and set the circular region with the center pixel position as the center and the radius r as the pixel pair sample sampling region, and set the pixels in the pixel pair sample sampling region belonging to the same class​​r and the center pixel x c P is attributed to the positive sample pixel pair set i , the pixel pair sample in the sample area belonging to different categories x r and the center pixel x c N is attributed to the negative sample pixel pair set i , wherein P i , N i (i={1, 2, 3, 4}) respectively represent the pseudo-labels obtained from the i-th stage of the network The positive and negative sample pixel pair sets P and N constructed in the step are used to construct the positive and negative sample pixel pair sets P and N in the network.

[0015] Step 303, extracting multi-layer decision pixel relationship, using the positive and negative sample pixel pair sets P i , N i (i={1, 2, 3, 4) constructed in different layers of the network to make decisions on the pixel relationship, if all P i (i={1, 2, 3, 4}) are determined as positive samples, then the final decision is a positive sample pixel pair, if there is any one group in N i (i={1, 2, 3, 4}) is determined as a negative sample, then the final decision is a negative sample pixel pair, and then the pixel relationship with higher reliability is extracted, and the multi-layer decision positive and negative pixel pair sets P and N are constructed.

[0016] Step four, constructing a pixel relationship network to obtain the centroid displacement vector and boundary feature of each pixel in the image.

[0017] Step 401, using the deep neural network ResNet-50 and two branches composed of 1x1 convolution to construct the pixel relationship network, and then inputting the original polarized SAR image into the pixel relationship network to obtain the centroid displacement vector D(x i of each pixel x i by one of the 1x1 convolution branches, and obtaining the boundary feature B(x i by the other 1x1 convolution branch.

[0018] Step five, training the pixel relationship network using the extracted pixel relationship information to obtain the pixel semantic affinity matrix.

[0019] Step 501, using the pixel pair relationship information in the multi-layer decision pixel relationship sets P and N to supervise the training of the centroid displacement vector D(x i ) and the boundary feature B(x i ) obtained by the pixel relationship network, and the network training loss function is loss=loss B +loss D , wherein loss B represents the boundary prediction training loss function, and loss DA displacement vector field training loss function is represented;

[0020] Step 502, boundary features B(x i ) extracted by the inter-pixel relationship network are utilized c The semantic affinity between pixel points x r is represented And a pixel semantic affinity matrix M aff is constructed c , wherein l(x r , x c ) represents the pixel position on the line connecting pixel points x r , B(x i ) represents the boundary feature of pixel x i , and max(·) represents the maximum value of the boundary feature prediction value;

[0021] Step six, pseudo-label optimization is performed using the pixel semantic affinity matrix;

[0022] Step 601, a probability transition matrix T is obtained using the pixel semantic affinity matrix M aff , and the expression of the probability transition matrix T is , wherein represents the matrix Hadamard product, γ represents the number of integrations, and is used to adjust the affinity value between pixels, is used to normalize ;

[0023] Step 602, the fused class activation map A fuse is optimized using the probability transition matrix, and the optimization process expression is , wherein is the optimized class activation map, θ represents the number of random walk iterations, B is the boundary prediction value, and R(·) represents the straightening operation on the matrix;

[0024] Step 603, the optimized class activation map is obtained according to the channel dimension to obtain the index corresponding to the maximum value, and the ground object category corresponding to the index is found to generate the optimized polarized SAR image pixel-level pseudo-label L ps ;

[0025] Step seven, the semantic segmentation network is trained using the optimized pixel-level pseudo-label, and the weakly supervised semantic segmentation of the polarized SAR image is completed;

[0026] Step 701, a semantic segmentation network composed of an encoder network and a decoder network is built, wherein the encoder network is composed of ResNet-50, and the decoder network is composed of a transposed convolutional neural network;

[0027] Step 702, input the original polarimetric SAR image into the semantic segmentation network to obtain a pixel-level prediction result

[0028] Step 703, using the optimized pixel-level pseudo label L ps The supervised semantic segmentation network is trained, and the network training loss is the cross entropy loss loss between the predicted value F out output by the semantic segmentation network and the pseudo label L ps . seg .

[0029] The SAR image weak supervised semantic segmentation method based on decision optimization pseudo label generation, characterized in that: the calculation formula of the multi-label classification loss for training the multi-label classification network in step 101 is as follows:

[0030]

[0031] Wherein, K represents the total number of samples, y(k) represents the label of the kth class in the image sample, if the image contains k categories, y(k) is equal to 1, and if the image does not contain k categories, y(k) is equal to 0, o(k) represents the prediction result output by the multi-label classification network, and σ(·) represents the Sigmoid activation function.

[0032] The SAR image weak supervised semantic segmentation method based on decision optimization pseudo label generation, characterized in that: the calculation formula of the class activation mapping result of the network shallow layer and the deep layer obtained by using the feature map and the back gradient of the last four stages of the network in step 102 is as follows:

[0033]

[0034] Wherein, A represents the class activation mapping, F i (i=1,…,4) represents the feature map extracted from the i th stage of the network, represents the channel weight corresponding to the c th category, for the class activation mapping of the network shallow layer, represents the feature map F i of the c th category.

[0035] The SAR image weak supervised semantic segmentation method based on decision optimization pseudo label generation, characterized in that: the calculation formula of the high confidence fusion class activation mapping A fuse in step 201 is as follows:

[0036] A fuse =(A1+A2+A3)×λ+A4

[0037] Wherein, λ represents an adjustable parameter, which is used to adjust the class activation mapping of different levels to A fusecontribution.

[0038] The above-mentioned weakly supervised semantic segmentation method of SAR images based on decision optimization pseudo-label generation is characterized in that: in step 302, a set of positive and negative sample pixel pairs P is constructed. i and N i The calculation formula is as follows:

[0039]

[0040]

[0041] Among them, x c represents the center pixel, x r Represents the pixels within the sample sampling area of ​​the pixel pair. Represents the pseudo label of the i-th stage of the network Medium pixels x c and x r The corresponding label, I g Represents the set of low-confidence pixel locations in the predicted pseudo-label.

[0042] The above-mentioned weakly supervised semantic segmentation method for SAR images based on decision-optimized pseudo-label generation is characterized in that the calculation formula for constructing the multi-layer decision positive and negative pixel pair sets P and N in step 303 is as follows:

[0043] P=P1∩P2∩P3∩P4

[0044] N=N1∪N2∪N3∪N4

[0045] Among them, P i 、N i (i=1,...,4) represent the pseudo labels The set of positive and negative sample pixel pairs obtained under .

[0046] The above-mentioned weakly supervised semantic segmentation method of SAR images based on decision optimization pseudo-label generation is characterized in that: in step 501, the boundary prediction training loss function loss B And the displacement vector field training loss function loss D The calculation formulas are as follows:

[0047]

[0048]

[0049] in, Represents pixel x c and x r The semantic affinity between c ,xr ) indicates that at pixel x c 、x r The pixel position on the line, B(x i ) represents pixel x i The boundary feature, max(·) means taking the maximum value of the boundary feature prediction value, Represents pixel x c and x r The position vector, D(x c )、D(x r ) represent pixel x c and x r The corresponding center of mass displacement vector.

[0050] The above-mentioned weakly supervised semantic segmentation method of SAR images based on decision optimization pseudo-label generation is characterized in that: the cross entropy loss loss of the semantic segmentation network in step 703 is seg The calculation formula is as follows:

[0051]

[0052] in, represents the pseudo-label prediction value, Represents the predicted value of the semantic segmentation network, K represents the number of target categories contained in the image, N = H × W × S represents the total number of pixels in the training sample, σ(·) represents the Softmax function, and S refers to the number of batches of network input.

[0053] Compared with the prior art, the present invention has the following advantages:

[0054] This method uses image-level labels of SAR images to train a multi-label classification network, fusing class activation maps from different layers of the network to generate pixel-level pseudo-labels with higher confidence. The network's multi-layer pseudo-label prediction results are used to make decisions about the relationships between image pixels, improving the reliability of the extracted pixel-level relationships and optimizing the pixel-level pseudo-labels of polarimetric SAR images. This method enables semantic segmentation of polarimetric SAR images under weakly supervised image-level annotation, alleviating the problem of insufficient pixel-level annotation in polarimetric SAR images. The method is simple in structure and easy to implement and use.

[0055] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 is a flow chart of the method of the present invention; DETAILED DESCRIPTION

[0057] The method of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments of the present invention.

[0058] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0059] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0060] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can, for example, be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0061] For ease of description, spatially relative terms such as "above", "above", "on the upper surface of", "above", etc. may be used herein to describe the spatial positional relationship of a device or feature to other devices or features as shown in the figures. It should be understood that spatially relative terms are intended to include different orientations of the device in use or operation in addition to the orientation described in the figures. For example, if the device in the drawings is inverted, the device described as "above other devices or structures" or "above other devices or structures" will be positioned as "below other devices or structures" or "below other devices or structures". Thus, the exemplary term "above" can include both "above" and "below". The device can also be positioned in other different ways (rotated 90 degrees or in other orientations), and the spatially relative descriptions used here are interpreted accordingly.

[0062] like Figure 1 As shown, the weakly supervised semantic segmentation method of SAR images based on decision optimization pseudo-label generation of the present invention is characterized by comprising the following steps:

[0063] Step one, training a multi-label classification network to obtain class activation mapping results of different layers of the neural network and pixel-level prediction pseudo labels;

[0064] Step 101, input the polarimetric SAR image and the corresponding image-level label into the multi-label classification network, and use the multi-label classification loss loss bce The network is trained, and the multi-label classification network uses ResNet-50 as the backbone network;

[0065] Step 102, from the trained multi-label classification network, use the feature maps of the last four stages of the network and the backpropagation gradient to obtain the class activation mapping results of the shallow and deep layers of the network Wherein, A1, A2, A3 represent the class activation mapping results of the shallow layers of the network, A4 represents the class activation mapping results of the deep layers of the network, H, W represent the length and width of the polarimetric SAR image respectively, and C represents the number of classes, which is also the channel dimension of the class activation mapping result. Specifically, the calculation formula for obtaining the class activation mapping results of the shallow and deep layers of the network using the feature maps of the last four stages of the network and the backpropagation gradient is as follows:

[0066]

[0067] Wherein, A represents the class activation mapping, F i (i = 1, …, 4) represents the feature map extracted from the i-th stage of the network, represents the channel weight corresponding to the c-th class, and for the class activation mapping of the shallow layers of the network, represents the feature map F i Gradient with respect to class c;

[0068] Step 103, for the class activation mapping results obtained by the network at different layers Each position in the formula returns the maximum value in the channel dimension C, and the channel index corresponding to the maximum value is taken as the predicted class, and then the pixel-level prediction pseudo label of the network at different layers is obtained Wherein, represents the prediction pseudo label obtained from the class activation mapping A i

[0069] Step two, fuse the class activation mapping results at different levels to obtain a class activation mapping result with higher confidence;

[0070] Step 201, fuse the class activation mapping results A1, A2, A3, A4 obtained by the network at different layers to generate a high-confidence fusion class activation mapping A fuse ; Specifically, the calculation formula of the high-confidence fusion class activation mapping A fuse

[0071] A​​fuse = (A1 + A2 + A3) * λ + A4

[0072] wherein λ represents an adjustable parameter, used to adjust the contribution degree of different levels of class activation map A fuse ;

[0073] In specific implementation, class activation maps A2 and A4 are selected to participate in generating the fusion class activation map result A fuse , so as to achieve the best pseudo-label generation result, and the adjustable parameter λ is set to 0.5;

[0074] Step three, using the pseudo-labels of different layers of the network to form a decision on the relationship between pixels, and extracting multi-layer decision pixel relationship information;

[0075] Step 301, according to the class activation map results A1, A2, A3 and A4, setting a high-confidence category area and a low-confidence area, and sorting the class activation map values corresponding to each category in the predicted pseudo-label , the value at the 30% position is set as a threshold, and the pixel positions greater than the threshold are set as the high-confidence category area, and the remaining pixel positions are set as the low-confidence area;

[0076] Step 302, constructing positive and negative sample pixel pair sets P i and N i , in the pseudo-label after setting the high-confidence category area and the low-confidence area, setting a center pixel position x c , and setting a circular area with the center pixel position as the center and a radius r as the pixel pair sample sampling area, the pixels x r and the center pixel x c belonging to the same category in the pixel pair sample sampling area are attributed to the positive sample pixel pair set P i , and the pixels x r and the center pixel x c belonging to different categories in the pixel pair sample sampling area are attributed to the negative sample pixel pair set N i , wherein P i , N i (i = {1, 2, 3, 4}) respectively represent the positive and negative sample pixel pair sets constructed from the pseudo-label obtained from the i-th stage of the network; specifically, the calculation formula of the positive and negative sample pixel pair sets P i and N i is as follows:

[0077]

[0078] ​

[0079] wherein x c represents the center pixel, x r represents the pixel pair sample sampling area within the pixel, respectively represent the pseudo label of the i-th stage of the network the label corresponding to the pixel x c and x r g indicates the set of low confidence pixel positions in the predicted pseudo label.

[0080] Step 303, extract the multi-layer decision pixel relationship, use the positive and negative sample pixel pair set P i , N i (i={1, 2, 3, 4}) to make a decision on the pixel relationship, if all P i (i={1, 2, 3, 4}) are identified as positive samples, then the final decision is a positive sample pixel pair, if there is any one group in N i (i={1, 2, 3, 4}) identified as negative sample, then the final decision is a negative sample pixel pair, and then extract the pixel relationship with higher reliability, and construct the multi-layer decision positive and negative pixel pair set P, N; Specifically, the calculation formula of the multi-layer decision positive and negative pixel pair set P, N is as follows:

[0081] P=P1∩P2∩P3∩P4

[0082] N=N1∪N2∪N3∪N4

[0083] wherein P i , N i (i=1,..., 4) respectively represent the positive and negative sample pixel pair set obtained under the pseudo label

[0084] In specific implementation, the initial positive and negative samples P3, N3, P4, N4 are selected to make a decision on the relationship between pixels, to ensure the reliability of the pixel relationship extracted from the pseudo label;

[0085] Step four, construct a pixel relationship network to obtain the centroid displacement vector and boundary feature of each pixel in the image;

[0086] Step 401, use the deep neural network ResNet-50 and two branches composed of 1x1 convolution to construct a pixel relationship network, and then input the original polarized SAR image into the pixel relationship network, and obtain the centroid displacement vector i of each pixel x by one of the 1x1 convolution branches, and obtain the boundary feature ​​

[0087] Step five, training the inter-pixel relationship network using the extracted pixel relationship information to obtain a pixel semantic affinity matrix;

[0088] Step 501, using the extracted multi-layer decision pixel relationship set P and N pixel pair relationship information to supervise the training of the centroid displacement vector D(x i ) and the boundary feature B(x i ) obtained by the inter-pixel relationship network, and the network training loss function is loss = loss B + loss D , wherein loss B represents the boundary prediction training loss function, and loss D represents the displacement vector field training loss function; specifically, the calculation formulas of the boundary prediction training loss function loss B and the displacement vector field training loss function loss D are as follows:

[0089]

[0090]

[0091] wherein, represents the semantic affinity between pixel points x c and x r , l(x c , x r ) represents the pixel point position on the line connecting pixel points x c and x r , B(x i ) represents the boundary feature of pixel x i , max(·) represents taking the maximum value of the boundary feature prediction value, respectively represent the position vectors of pixel points x c and x r , D(x c ) and D(x r ) respectively represent the centroid displacement vectors corresponding to pixel points x c and x r .

[0092] Step 502, using the boundary feature B(x i ) extracted by the inter-pixel relationship network to represent the semantic affinity between pixel points x c and x r and constructing a pixel semantic affinity matrix M aff , wherein l(x c , x r ) represents the pixel point position on the line connecting pixel points x c and x​r pixel position on the connection, B(x i ) represents the boundary feature of pixel x i , max(·) represents the maximum value of the boundary feature prediction value;

[0093] Step six, using the pixel semantic affinity matrix to optimize the pseudo label;

[0094] Step 601, using the pixel semantic affinity matrix M aff , to obtain the probability transition matrix T, and the expression of the probability transition matrix T is wherein, represents the matrix Hadamard product, γ represents the integral number of times, used to adjust the affinity value between pixels, used to normalize ;

[0095] In specific implementation, the integral number of times γ used to calculate the probability transition matrix is set to 5;

[0096] Step 602, using the probability transition matrix to optimize the fusion class activation map A fuse , and the optimization process expression is wherein, is the optimized class activation map, θ represents the random walk iteration number, B is the boundary prediction value, and R(·) represents the straightening operation on the matrix;

[0097] In specific implementation, the random walk iteration number θ is set to 6;

[0098] Step 603, obtaining the index corresponding to the maximum value according to the channel dimension for the class activation map after optimization, and finding the ground object category corresponding to the index to generate the optimized polarimetric SAR image pixel-level pseudo label L ps ;

[0099] Step seven, using the optimized pixel-level pseudo label to train the semantic segmentation network to complete the weakly supervised semantic segmentation of the polarimetric SAR image;

[0100] Step 701, building a semantic segmentation network composed of an encoder network and a decoder network, wherein the encoder network is composed of ResNet-50, and the decoder network is composed of a transposed convolutional neural network;

[0101] Step 702, inputting the original polarimetric SAR image into the semantic segmentation network to obtain the pixel-level prediction result

[0102] Step 703, using the optimized pixel-level pseudo label L psSupervise the semantic segmentation network for training, and the network training loss is the predicted value F output by the semantic segmentation network out and pseudo-label L ps The cross entropy loss between seg ; Cross entropy loss of the semantic segmentation network in step 703 seg The calculation formula is as follows:

[0103]

[0104] in, represents the pseudo-label prediction value, represents the predicted value of the semantic segmentation network, K represents the number of target categories contained in the image, N = H × W × S represents the total number of pixels in the training sample, σ(·) represents the Softmax function, and S refers to the number of batches of network input.

[0105] The above description is merely an embodiment of the present invention and does not limit the present invention in any way. Any simple modification, change and equivalent structural change made to the above embodiment based on the technical essence of the present invention shall still fall within the scope of protection of the technical solution of the present invention.

Claims

1. A SAR image weakly supervised semantic segmentation method based on decision optimization pseudo-label generation, characterized in that: Comprising the following steps: Step one, training a multi-label classification network, obtaining class activation mapping results of different layers of the neural network and pixel-level prediction pseudo-labels; Step 101, input the polarimetric SAR image and the corresponding image-level label into the multi-label classification network, and use the multi-label classification loss loss bce The network is trained, and the multi-label classification network uses ResNet-50 as a backbone network; Step 102, obtaining class activation mapping results of the shallow layer and the deep layer of the network from the trained multi-label classification network by using the feature maps of the last four stages of the network and the back gradient wherein A1, A2, A3 represent the class activation mapping results of the shallow layer of the network, A4 represents the class activation mapping result of the deep layer of the network, H and W represent the length and width of the polarized SAR image respectively, and C represents the number of categories, which is also the channel dimension of the class activation mapping result. Step 103, obtaining class activation map results obtained by different layers of the network Each position in the class activation map A is returned as a maximum value in the channel dimension C, and the channel index corresponding to the maximum value is taken as a predicted class, thereby obtaining pixel-level prediction pseudo-labels of different layers of the network wherein, represents a prediction pseudo-label obtained from the class activation map A i . Step two, fuse class activation mapping results of different layers to obtain class activation mapping results with higher confidence; Step 201, the class activation map results A1, A2, A3, A4 obtained by different layers of the fusion network are fused to generate a class activation map A with high confidence fuse ; Step three, use the pseudo-labels of different layers of the network to form a decision on the relationship between pixels, and extract multi-layer decision pixel relationship information; Step 301, obtaining pseudo labels for different layers of the network according to the class activation map results A1, A2, A3 and A4 Setting a high-confidence class region and a low-confidence region, and predicting pseudo labels Sorting the class activation map values corresponding to each class in each region, setting the value at the 30% position as a threshold, screening out pixel positions greater than the threshold as the high-confidence class region, and setting the remaining pixel positions as the low-confidence region. Step 302: Construct a set of positive and negative sample pixel pairs P i and N i , the pseudo label after setting the high confidence category area and low confidence area , set the center pixel position x c , and set the circular area with the central pixel position as the center and the radius r as the pixel pair sample sampling area, and set the pixels x in the pixel pair sample sampling area belonging to the same category r With the center pixel x c Belong to the positive sample pixel pair set P i , pixels belonging to different categories are sampled to pixels x in the sampling area r With the center pixel x c Attributed to the negative sample pixel pair set N i , where P i 、N i (i={1,2,3,4}) represents the pseudo labels obtained from the i-th stage of the network The set of positive and negative sample pixel pairs constructed in; Step 303, extracting multi-layer decision pixel relationship, using the positive and negative sample pixel pair set P constructed in different layers of the network i , N i (i={1, 2, 3, 4}) to make a decision on the pixel relationship, if all P i (i={1, 2, 3, 4}) are identified as positive samples, then the final decision is a positive sample pixel pair, if any N i (i={1, 2, 3, 4}) is identified as a negative sample, then the final decision is a negative sample pixel pair, and then the pixel relationship with higher reliability is extracted, and the multi-layer decision positive and negative pixel pair set P and N is constructed; Step four, construct a pixel relationship network to obtain the centroid displacement vector and boundary feature of each pixel in the image; Step 401, a pixel relationship network is constructed by using a deep neural network ResNet-50 and two branches composed of 1x1 convolution, and then the original polarimetric SAR image is input into the pixel relationship network, and the centroid displacement vector of each pixel x i is obtained from one of the 1x1 convolution branches Boundary features are obtained from the other 1x1 convolution branch Step five, use the extracted pixel relationship information to train the pixel relationship network to obtain the pixel semantic affinity matrix; Step 501, supervise the centroid displacement vector D(x i ) and the boundary feature B(x i ) obtained by the pixel relationship network with the extracted pixel pair relationship information in the multi-layer decision pixel relationship set P and N, and the network training loss function is loss = loss B + loss D , wherein, loss B represents the boundary prediction training loss function, and loss D represents the displacement vector field training loss function; Step 502: Use the boundary feature B(x i ) represents pixel x c and x r Semantic affinity between And construct the pixel semantic affinity matrix M aff , where l(x c ,x r ) indicates that at pixel x c 、x r The pixel position on the line, B(x i ) represents pixel x i The boundary feature of , max(·) means taking the maximum value of the boundary feature prediction value; Step six, use the pixel semantic affinity matrix to optimize the pseudo-labels; Step 601, utilizing the pixel semantic affinity matrix M aff , obtaining a probability transition matrix T, and an expression of the probability transition matrix T is wherein, represents a matrix Hadamard product, γ represents a product number, and is used for adjusting the affinity value between pixels, for normalizing ; Step 602, using the probability transition matrix to fuse the class activation map A fuse Optimization is performed, and the optimization process expression is wherein, is the optimized class activation map, θ represents the number of iterations of random walk, B is the boundary prediction value, and R(·) represents a straightening operation on a matrix; Step 603, performing optimization on the class activation map According to the channel dimension, an index corresponding to the maximum value is obtained, and a ground object class corresponding to the index is searched to generate an optimized polarimetric SAR image pixel-level pseudo label L ps ; Step seven, use the optimized pixel-level pseudo-labels to train the semantic segmentation network to complete the weakly supervised semantic segmentation of the polarized SAR image; Step 701, build a semantic segmentation network composed of an encoder network and a decoder network, wherein the encoder network is composed of ResNet-50, and the decoder network is composed of a transposed convolutional neural network; Step 702, input the original polarimetric SAR image into the semantic segmentation network to obtain a pixel-level prediction result Step 703, using the optimized pixel-level pseudo label L ps The supervised semantic segmentation network is trained, and the network training loss is the cross entropy loss loss between the predicted value F output by the semantic segmentation network and the pseudo label L out ps seg .​​ 2. The SAR image weakly supervised semantic segmentation method based on decision optimization pseudo-label generation according to claim 1, characterized in that: The calculation formula of the multi-label classification loss used to train the multi-label classification network in step 101 is as follows: Wherein, K represents the total number of samples, y(k) represents the label of the kth class in the image sample, if there is k class in the image, y(k) is equal to 1, otherwise, y(k) is equal to 0, o(k) represents the prediction result output by the multi-label classification network, and sigma(·) represents the Sigmoid activation function.

3. The SAR image weakly supervised semantic segmentation method based on decision optimization pseudo-label generation according to claim 1, characterized in that: The calculation formula of the class activation mapping results of the shallow and deep layers of the network in step 102 using the feature maps and the back propagation gradient of the last four stages of the network is as follows: where A denotes the class activation map, F i (i = 1, …, 4) denotes the feature map extracted from the i-th stage of the network, denotes the channel weight corresponding to the c-th class, for the class activation map of the shallow layer of the network, denotes the feature map F i the gradient with respect to the class c.

4. The SAR image weakly supervised semantic segmentation method based on decision optimization pseudo-label generation according to claim 1, characterized in that: The high confidence fusion class activation map A in step 201 fuse The calculation formula is as follows: A fuse = (A1 + A2 + A3) x lambda + A4 where λ represents an adjustable parameter for adjusting the contribution of different hierarchical class activation map pairs A fuse to the final prediction.

5. The SAR image weakly supervised semantic segmentation method based on decision optimization pseudo-label generation according to claim 1, characterized in that: The set P of positive and negative sample pixel pairs is constructed in step 302 i and N i The calculation formula is as follows: wherein x c represents the center pixel, x r represents the pixel pair sample sampling region within the pixel, respectively represent the pseudo label of the i-th stage of the network the label corresponding to the pixel x c and x r in the middle, I g represents the set of low confidence pixel positions in the predicted pseudo label.

6. The SAR image weakly supervised semantic segmentation method based on decision optimization pseudo-label generation according to claim 1, characterized in that: The calculation formula of the positive and negative pixel pair set P and N in step 303 is as follows: P=P1∩P2∩P3∩P4 N=N1∪N2∪N3∪N4 where P i , N i (i = 1,..., 4) represent the positive and negative sample pixel pair sets obtained under the pseudo labels ​ 7. The SAR image weakly supervised semantic segmentation method based on decision optimization pseudo-label generation according to claim 1, characterized in that: Boundary prediction training loss function loss in step 501 B and displacement vector field training loss function loss D The calculation formulas are as follows, respectively. wherein, represents the semantic affinity between pixel points x c and x r , l(x c , x r ) represents the pixel position on the line connecting pixel points x c , x r , B(x i ) represents the boundary feature of pixel x i , and max(·) represents the maximum value of the boundary feature prediction value, respectively represent the position vectors of pixel points x c and x r , and D(x c ), D(x r ) respectively represent the corresponding centroid displacement vectors of pixel points x c and x r .

8. The SAR image weakly supervised semantic segmentation method based on decision optimization pseudo-label generation according to claim 1, characterized in that: The cross-entropy loss loss of the semantic segmentation network in step 703 seg The calculation formula is as follows: wherein, denotes a pseudo-label prediction value, denotes a semantic segmentation network prediction value, K denotes a target class number contained in an image, N = H x W x S denotes a total number of pixel points of a training sample, σ(·) denotes a Softmax function, and S denotes a batch number of network input.

Images