Unsupervised domain adaptation polarimetric sar feature classification method based on cross-domain feature fusion

Abstract

The application discloses a kind of unsupervised domain adaptation polarimetric SAR ground feature classification methods based on cross-domain feature fusion, belong to image processing technical field, including: obtaining the polarimetric SAR image to be handled, and it is polarized coherent decomposition, obtain the polarimetric decomposition feature of the polarimetric SAR image;The polarimetric decomposition feature of polarimetric SAR image is input into pre-trained ground feature classification model, so that ground feature classification model is based on the polarimetric decomposition feature of polarimetric SAR image and the first polarimetric decomposition feature of source domain sample obtained in the training process of ground feature classification model, respectively carry out the domain feature fusion of target domain and source domain, further to the target domain fusion feature and source domain fusion feature obtained are domain feature alignment, obtain the final fusion feature after, based on the final fusion feature output ground feature classification result.The application guarantees classification precision while solving the problem of label shift.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically relating to an unsupervised domain adaptive polarimetric SAR land cover classification method based on cross-domain feature fusion. Background Technology

[0002] Polarimetric Synthetic Aperture Radar (PolSAR) is characterized by its all-weather and all-time operation, providing abundant scattering information and finding wide application in environmental monitoring, resource exploration, and vegetation change detection. With the increasing number of available PolSAR sensors, a large amount of PolSAR data is available for processing and analysis. However, the scarcity of labeled data and domain offset in PolSAR images degrade the performance of supervised learning algorithms. Therefore, unsupervised domain adaptation (UDA) for PolSAR terrain classification is needed.

[0003] Unsupervised domain adaptation (UDA) is a subfield of machine learning that focuses on transferring knowledge from one domain to another. UDA has two main types of domain shifts: label shift and feature distribution shift. Label shift occurs when the proportion of each class differs between the source and target domains, while feature distribution shift describes the difference in feature distributions between the source and target domains. In polarimetric synthetic aperture radar (PSAR) terrain classification tasks, the causes of distribution differences are complex and varied, including variations in sensors, imaging conditions, imaging time, and imaging area. Furthermore, there are significant differences between PSAR images and natural images. These factors lead to feature distribution shifts and label shifts between different PSAR images.

[0004] Currently, for polarimetric synthetic aperture radar (PolarisSAR) terrain classification tasks, existing technologies fall into two categories: the first is based on manually designed features, which focuses on manually designing features to reduce feature distribution offset between the source and target domains; the second is based on class feature alignment, which first generates pseudo-labels for the target domain and then uses the domain discriminator of the generated pseudo-labels to align the class feature distributions between the source and target domains. Both methods emphasize the feature distribution offset problem in UDA while neglecting the label offset problem. Furthermore, the class feature alignment-based UDA method uses pseudo-labels from the target domain to achieve class feature alignment, inevitably introducing label noise during training and degrading UDA performance. Summary of the Invention

[0005] To address the aforementioned problems in existing technologies, this invention provides an unsupervised domain-adaptive polarimetric SAR land cover classification method based on cross-domain feature fusion. The technical problem to be solved by this invention is achieved through the following technical solution:

[0006] This invention provides an unsupervised domain-adaptive polarimetric SAR land cover classification method based on cross-domain feature fusion, comprising:

[0007] The polarimetric SAR image to be processed is acquired and subjected to polarimetric coherent decomposition to obtain the polarimetric decomposition features of the polarimetric SAR image.

[0008] The polarization decomposition features of the polarimetric SAR image are input into a pre-trained land cover classification model. The land cover classification model performs intra-domain feature fusion of the target domain and the source domain based on the polarization decomposition features of the polarimetric SAR image and the first polarization decomposition features of the source domain samples obtained during the training process. The obtained target domain fusion features and source domain fusion features are further aligned with inter-domain features to obtain the final fusion features. The land cover classification result is then output based on the final fusion features.

[0009] In one embodiment of the present invention, the land cover classification model is trained using the following steps:

[0010] Obtain a first preset number of source domain samples and a second preset number of target domain samples;

[0011] After performing polarimetric coherent decomposition on the source domain samples and the target domain samples, the first polarimetric decomposition feature of each source domain sample and the second polarimetric decomposition feature of each target domain sample are obtained.

[0012] The source domain samples, the target domain samples, the first polarization decomposition features, and the second polarization decomposition features are input into the land cover classification model to be trained; the land cover classification model to be trained includes: a domain balancing sampling module and a cross-domain feature fusion module, and the cross-domain feature fusion module includes: a shallow feature extractor and a cross-fusion layer;

[0013] Based on the source domain samples, the target domain samples, the first polarization decomposition feature, and the second polarization decomposition feature, a domain-balanced training dataset is constructed using a domain-balanced sampling module. The domain-balanced training dataset includes a source domain training dataset and a target domain training dataset. The source domain training dataset includes source domain training data of different source domain categories, and the target domain training dataset includes target domain training data of different types of pseudo-labels.

[0014] A third preset number of source domain training data and target domain training data are obtained respectively, and the first shallow feature of each source domain training data and the second shallow feature of each target domain training data are extracted using the shallow feature extractor.

[0015] The first shallow features of the training data from each source domain and the second shallow features of the training data from each target domain are input into the cross-fusion layer. The final fused features are obtained through intra-domain feature fusion and inter-domain feature alignment.

[0016] The final fused features are input into the fully connected layer to obtain the predicted probabilities of the target domain and the source domain.

[0017] The total loss value is calculated based on the predicted probability of the target domain, the predicted probability of the source domain, the ground truth value of the source domain training data, and the pseudo-label of the target domain training data. Backpropagation is then performed based on the total loss value to adjust the parameters of the land cover classification model to be trained until the preset conditions are met.

[0018] In one embodiment of the present invention, the step of constructing a domain-balanced training dataset using a domain-balanced sampling module based on the source domain samples, the target domain samples, the first polarization decomposition feature, and the second polarization decomposition feature includes:

[0019] Based on the first preset number of source domain samples, calculate the cluster center for each source domain category;

[0020] For each target domain sample, calculate the Wishart distance between each pixel and the cluster center of each source domain category, and determine the pseudo-label of each pixel based on the source domain category corresponding to the smallest Wishart distance;

[0021] The first polarization decomposition feature of each source domain sample is cropped into multiple first sub-blocks. After obtaining the same number of first sub-blocks under each source domain category through random sampling, each first sub-block is used as source domain training data to form the source domain training dataset.

[0022] The second polarization decomposition features of each target domain sample are cropped into multiple second sub-blocks. After obtaining the same number of second sub-blocks with the same pseudo-labels through random sampling, each second sub-block is used as the target domain training data to form the target domain training dataset.

[0023] A domain-balanced training dataset is formed based on the source domain training dataset and the target domain training dataset.

[0024] In one embodiment of the present invention, the cross-fusion layer includes a self-fusion unit and a cross-fusion unit;

[0025] The steps of inputting the first shallow features of the training data from each source domain and the second shallow features of the training data from each target domain into the cross-fusion layer, and obtaining the final fused features through intra-domain feature fusion and inter-domain feature alignment, include:

[0026] The first shallow features of the training data from each source domain and the second shallow features of the training data from each target domain are respectively input into the self-fusion unit;

[0027] The self-fusion unit generates an undirected graph of the source domain based on the first shallow features and generates an undirected graph of the target domain based on the second shallow features.

[0028] Intra-domain feature fusion is performed based on the source domain undirected graph and the target domain undirected graph respectively to obtain source domain fusion features and target domain fusion features;

[0029] Inter-domain feature alignment is performed based on the source domain fusion features and the target domain fusion features to obtain the final fusion features.

[0030] In one embodiment of the present invention, the self-fusion unit generates the source domain undirected graph and the target domain undirected graph according to the following steps:

[0031] The first shallow features of each source domain training data are used as nodes of the source domain undirected graph, and the query matrix Q, key matrix K and value matrix V of each node in the source domain undirected graph are calculated respectively.

[0032] Based on the query matrix Q, key matrix K, and value matrix V of each node in the source domain undirected graph, calculate the edge value of the edge connecting two adjacent nodes in the source domain undirected graph to generate the source domain undirected graph.

[0033] The second shallow features of the training data of each target domain are used as nodes of the undirected graph of the target domain, and the query matrix Q, key matrix K and value matrix V of each node in the undirected graph of the target domain are calculated respectively.

[0034] Based on the query matrix Q, key matrix K, and value matrix V of each node in the undirected graph of the target domain, calculate the edge value of the edge connecting two adjacent nodes in the undirected graph of the target domain, and generate the undirected graph of the target domain.

[0035] In one embodiment of the present invention, the query matrix Q, key matrix K, and value matrix V of the i-th node in the source domain undirected graph and the target domain undirected graph are represented as follows:

[0036]

[0037] Among them, Q i K i V i Let Q, K, and V represent the query matrix, key matrix, and value matrix of the i-th node in the source domain undirected graph and the target domain undirected graph, respectively. W1, W2, and W3 represent the preset weight matrices, and b1, b2, and b3 represent the preset biases.

[0038] In an undirected graph with both source and target domains, the edge value of the edge connecting the i-th node and the j-th node is represented as follows:

[0039]

[0040] Where T represents the transpose operation, Softmax() represents the Softmax activation function, and α ij Let represent the edge value of the edge connecting the i-th node and the j-th node in the undirected graph of the source domain and the undirected graph of the target domain, where i ≠ j.

[0041] In one embodiment of the present invention, the step of fusing intra-domain features based on the source domain undirected graph and the target domain undirected graph respectively to obtain source domain fused features and target domain fused features includes:

[0042] For each node in the source domain undirected graph, its corresponding first shallow feature is fused with the first shallow features of all nodes in the source domain undirected graph to obtain the source domain fused feature of that node.

[0043] For each node in the target domain undirected graph, its corresponding second shallow feature is fused with the second shallow features corresponding to all nodes in the target domain undirected graph to obtain the target domain fused feature of that node.

[0044] The source domain fusion feature and target domain fusion feature of the i-th node in the source domain undirected graph and the target domain undirected graph are represented as follows:

[0045] z i =∑α in v n ;

[0046] Among them, v n Let α represent the first shallow feature and the second shallow feature of the nth node in the undirected graph of the source domain and the undirected graph of the target domain, where n = 1, 2, ..., N, and N represents the number of nodes in the undirected graph of the source domain and the undirected graph of the target domain. The number of nodes in the undirected graph of the source domain and the undirected graph of the target domain is the same as the number of training data in the source domain and the target domain, respectively. in Let z represent the edge value of the edge connecting the i-th node and the n-th node in the undirected graph of the source domain and the undirected graph of the target domain. i This represents the source domain fusion feature and target domain fusion feature of the i-th node in the source domain undirected graph and the target domain undirected graph.

[0047] In one embodiment of the present invention, the step of performing inter-domain feature alignment based on source domain fusion features and target domain fusion features to obtain the final fusion features includes:

[0048] The source domain fusion feature of each node in the source domain undirected graph is taken as the first node, and the target domain fusion feature of each node in the target domain undirected graph is taken as the second node. The edge value of the cross edge connecting the first node and the second node is calculated to construct an undirected graph to represent the relationship between the source domain and the target domain.

[0049] For each first node and second node in the undirected graph representing the relationship between the source domain and the target domain, the corresponding source domain fusion feature and target domain fusion feature are respectively fused with the target domain fusion feature and source domain fusion feature of itself and the adjacent second node and first node to obtain the final fusion feature corresponding to each first node and second node.

[0050] In one embodiment of the present invention, the step of inputting the final fused features into a fully connected layer to obtain the predicted probability of the target domain and the predicted probability of the source domain includes:

[0051] The final fusion feature corresponding to the first node is input into the fully connected layer to obtain the prediction probability of the source domain, and the final fusion feature corresponding to the second node is input into the fully connected layer to obtain the prediction probability of the target domain.

[0052] The steps include: calculating a loss value based on the predicted probability of the target domain, the predicted probability of the source domain, the ground truth value of the source domain training data, and the pseudo-label of the target domain training data; performing backpropagation based on the loss value to adjust the parameters of the land cover classification model to be trained until preset conditions are met; and so on.

[0053] Calculate the loss value of the source domain classification loss function based on the predicted probability of the source domain and the true value of the source domain training data;

[0054] Calculate the loss value of the target domain classification loss function based on the predicted probability of the target domain and the pseudo-labels of the target domain training data;

[0055] The loss values ​​of the source domain classification loss function and the target domain classification loss function are weighted and summed to obtain the total loss value. Then, backpropagation is performed based on the loss value to adjust the parameters of the land cover classification model to be trained until the preset conditions are met.

[0056] In one embodiment of the present invention, the source domain classification loss function is expressed as:

[0057]

[0058] In the formula, y nc This represents the true value of the nth source domain training data belonging to the cth class, where c = 1, 2, ..., C, and C represents the number of classes. nc This represents the predicted probability that the nth source domain training data belongs to the cth category;

[0059] The target domain classification loss function is expressed as follows:

[0060]

[0061] In the formula, The pseudo-label of the nth source domain training data belongs to the cth class. This represents the predicted probability that the nth target domain training data belongs to the cth category.

[0062] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0063] This invention provides an unsupervised domain-adaptive polarimetric SAR land cover classification method based on cross-domain feature fusion. Since the land cover classification model is designed for the polarimetric decomposition features of polarimetric SAR images, it can combine the first polarimetric decomposition features of the source domain samples obtained during training to perform intra-domain feature fusion of the target domain and the source domain respectively, and perform inter-domain feature alignment on the obtained target domain fusion features and source domain fusion features, thus effectively alleviating the label shift problem.

[0064] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0065] Figure 1 This is a flowchart of the unsupervised domain adaptive polarimetric SAR land cover classification method based on cross-domain feature fusion provided in this embodiment of the invention;

[0066] Figure 2 This is a schematic diagram of the land cover classification model provided in an embodiment of the present invention;

[0067] Figure 3 This is a schematic diagram of intra-domain feature fusion and inter-domain feature alignment provided in an embodiment of the present invention. Detailed Implementation

[0068] The present invention will be further described in detail below with reference to specific embodiments, but the implementation of the present invention is not limited thereto.

[0069] Figure 1 This is a flowchart of an unsupervised domain adaptive polarimetric SAR land cover classification method based on cross-domain feature fusion provided in an embodiment of the present invention. Please refer to [link / reference]. Figure 1 This invention provides an unsupervised domain adaptive polarimetric SAR land cover classification method based on cross-domain feature fusion, comprising:

[0070] S11. Obtain the polarimetric SAR image to be processed and perform polarimetric coherent decomposition on it to obtain the polarimetric decomposition features of the polarimetric SAR image.

[0071] S12. Input the polarization decomposition features of the polarimetric SAR image into the pre-trained land cover classification model. The land cover classification model performs intra-domain feature fusion of the target domain and the source domain based on the polarization decomposition features of the polarimetric SAR image and the first polarization decomposition features of the source domain samples obtained during the training process. Then, the inter-domain feature alignment is performed on the obtained target domain fusion features and source domain fusion features to obtain the final fusion features. Finally, the land cover classification result is output based on the final fusion features.

[0072] It should be noted that in step S11, various methods can be used to perform polarimetric coherent decomposition on the polarimetric SAR image to be processed, such as coherence matrix T, decomposition methods based on scattering models (Freeman-Durden decomposition, Yamaguchi decomposition), coherent decomposition methods based on scattering matrix S (Pauli decomposition, Krogager decomposition, Cameron decomposition), and decomposition methods based on eigenvalues ​​or eigenvectors (H / A / Alpha decomposition, cloude decomposition, Holm decomposition). After processing the polarimetric SAR image to be processed using these methods in sequence, the polarimetric decomposition features obtained by stitching together can be used to obtain the polarimetric decomposition features of the polarimetric SAR image to be processed.

[0073] Figure 2 This is a schematic diagram of the land cover classification model provided in an embodiment of the present invention. For example... Figure 2 As shown, in this embodiment, the land cover classification model can be trained using the following steps:

[0074] S21. Obtain a first preset number of source domain samples and a second preset number of target domain samples;

[0075] S22. After performing polarization coherent decomposition on the source domain samples and target domain samples, the first polarization decomposition feature of each source domain sample and the second polarization decomposition feature of each target domain sample are obtained.

[0076] S23. Input the source domain samples, target domain samples, first polarization decomposition features and second polarization decomposition features into the land cover classification model to be trained; the land cover classification model to be trained includes: a domain balancing sampling module and a cross-domain feature fusion module, and the cross-domain feature fusion module includes: a shallow feature extractor and a cross-fusion layer;

[0077] S24. Based on source domain samples, target domain samples, first polarization decomposition features, and second polarization decomposition features, construct a domain-balanced training dataset using the domain-balanced sampling module. The domain-balanced training dataset includes a source domain training dataset and a target domain training dataset. The source domain training dataset includes source domain training data of different source domain categories, and the target domain training dataset includes target domain training data of different types of pseudo-labels.

[0078] S25. Obtain a third preset number of source domain training data and target domain training data respectively, and use a shallow feature extractor to extract the first shallow feature of each source domain training data and the second shallow feature of each target domain training data.

[0079] S26. Input the first shallow features of the training data of each source domain and the second shallow features of the training data of each target domain into the cross-fusion layer. Through intra-domain feature fusion and inter-domain feature alignment, the final fused features are obtained.

[0080] S27. Input the final fused features into the fully connected layer to obtain the predicted probability of the target domain and the predicted probability of the source domain.

[0081] S28. Calculate the total loss value based on the predicted probability of the target domain, the predicted probability of the source domain, the true value of the source domain training data, and the pseudo-label of the target domain training data. Perform backpropagation based on the total loss value to adjust the parameters of the land cover classification model to be trained until the preset conditions are met.

[0082] For example, please see Figure 2 I S G represents the source domain sample. S I represents the set of labels for the source domain samples. T X represents the target domain sample. S X represents the source domain training set, containing a third preset number of source domain training data. T The target domain training set contains a third preset number of target domain training data. SFE (Shallow Feature Extractor) represents a shallow feature extractor, and COF (Cross-over Fusion) represents a cross-fusion layer. In this embodiment, the source domain samples and target domain samples can be polarimetric SAR images collected in different seasons or at different times. After performing polarimetric coherence decomposition on the source domain samples and target domain samples, the first polarimetric decomposition feature of each source domain sample and the second polarimetric decomposition feature of each target domain sample can be obtained. Then, the first polarimetric decomposition feature, the second polarimetric decomposition feature, along with the source domain samples and the target domain samples, are used as input to the land cover classification model to be trained. Optionally, the land cover classification model to be trained includes a domain balancing sampling module and a cross-domain feature fusion module. The domain balancing sampling module is used to construct a domain balancing training dataset composed of the source domain training dataset and the target domain training dataset. The cross-domain feature fusion module is used to extract shallow features from the training data and the target domain training data, and then sequentially perform intra-domain feature fusion and inter-domain feature alignment based on the shallow features to obtain the final fused features.

[0083] Specifically, the steps for constructing a domain-balanced training dataset using the domain-balanced sampling module, based on source domain samples, target domain samples, first polarization decomposition features, and second polarization decomposition features, include:

[0084] Based on a first preset number of source domain samples, calculate the cluster center for each source domain category;

[0085] For each target domain sample, calculate the Wishart distance between each pixel and the cluster center of each source domain category, and determine the pseudo-label of each pixel based on the source domain category corresponding to the smallest Wishart distance;

[0086] The first polarization decomposition feature of each source domain sample is cropped into multiple first sub-blocks. After obtaining the same number of first sub-blocks under each source domain category through random sampling, each first sub-block is used as source domain training data to form the source domain training dataset.

[0087] The second polarization decomposition features of each target domain sample are cropped into multiple second sub-blocks. After obtaining the same number of second sub-blocks with the same pseudo-labels through random sampling, each second sub-block is used as the target domain training data to form the target domain training dataset.

[0088] A domain-balanced training dataset is formed based on the source domain training dataset and the target domain training dataset.

[0089] It should be understood that each source domain sample may include targets of multiple source domain categories, such as buildings, fields, and streams. Therefore, the cluster center of each source domain category can be calculated by combining all source domain samples. Then, the Wishart distance between each pixel in each target domain sample and the cluster center of each source domain category is calculated, and the source domain category corresponding to the smallest Wishart distance is used as the pseudo-label of the pixel. Further, the first polarization decomposition features of the source domain samples and the second polarization decomposition features of the target domain samples are respectively cropped into 15×15 first and second sub-blocks. The same number of first sub-blocks under each source domain category are obtained by random sampling to form the source domain training dataset. At the same time, the same number of second sub-blocks under each pseudo-label are obtained by random sampling to form the target domain training dataset.

[0090] In this embodiment, the cross-domain feature fusion module includes a shallow feature extractor and a cross-fusion layer. The cross-fusion layer further includes a self-fusion unit and a cross-fusion unit. The shallow feature extractor includes a BatchNormalization layer, a convolutional layer, a ReLU activation layer and a global average pooling layer connected in sequence, used to extract the first shallow features of the source domain training data and the second shallow features of the target domain training data.

[0091] Optionally, step S26, which involves inputting the first shallow features of the training data from each source domain and the second shallow features of the training data from each target domain into the cross-fusion layer, and obtaining the final fused features through intra-domain feature fusion and inter-domain feature alignment, includes:

[0092] The first shallow features of the training data from each source domain and the second shallow features of the training data from each target domain are respectively input into the self-fusion unit;

[0093] The source domain undirected graph is generated based on the first shallow feature using the self-fusion unit, and the target domain undirected graph is generated based on the second shallow feature.

[0094] Intra-domain feature fusion is performed based on the source domain undirected graph and the target domain undirected graph respectively to obtain source domain fusion features and target domain fusion features;

[0095] Inter-domain feature alignment is performed based on the source domain fusion features and the target domain fusion features to obtain the final fusion features.

[0096] In this embodiment, the self-fusion unit generates the source domain undirected graph and the target domain undirected graph according to the following steps:

[0097] The first shallow features of each source domain training data are used as nodes of the source domain undirected graph, and the query matrix Q, key matrix K and value matrix V of each node in the source domain undirected graph are calculated respectively.

[0098] Based on the query matrix Q, key matrix K, and value matrix V of each node in the source domain undirected graph, calculate the edge value of the edge connecting two adjacent nodes in the source domain undirected graph to generate the source domain undirected graph.

[0099] The second shallow features of the training data of each target domain are used as nodes of the undirected graph of the target domain. The query matrix Q, key matrix K and value matrix V of each node in the undirected graph of the target domain are calculated respectively.

[0100] Based on the query matrix Q, key matrix K, and value matrix V of each node in the undirected graph of the target domain, calculate the edge value of the edge connecting two adjacent nodes in the undirected graph of the target domain, and generate the undirected graph of the target domain.

[0101] The query matrix Q, key matrix K, and value matrix V of the i-th node in the source domain undirected graph and the target domain undirected graph are represented as follows:

[0102]

[0103] Among them, Q i K i V i Let Q, K, and V represent the query matrix, key matrix, and value matrix of the i-th node in the source domain undirected graph and the target domain undirected graph, respectively. W1, W2, and W3 represent the preset weight matrices, and b1, b2, and b3 represent the preset biases.

[0104] In an undirected graph with both source and target domains, the edge value of the edge connecting the i-th node and the j-th node is represented as follows:

[0105]

[0106] Where T represents the transpose operation, Softmax() represents the Softmax activation function, and α ij Let represent the edge value of the edge connecting the i-th node and the j-th node in the undirected graph of the source domain and the undirected graph of the target domain, where i ≠ j.

[0107] Step S26, which involves fusing intra-domain features based on the source domain undirected graph and the target domain undirected graph respectively to obtain source domain fused features and target domain fused features, includes:

[0108] For each node in the source domain undirected graph, its corresponding first shallow feature is fused with the first shallow features of all nodes in the source domain undirected graph to obtain the source domain fused feature of that node.

[0109] For each node in the undirected graph of the target domain, its corresponding second shallow feature is fused with the second shallow features of all nodes in the undirected graph of the target domain to obtain the target domain fused feature of that node.

[0110] The source domain fusion feature and target domain fusion feature of the i-th node in the source domain undirected graph and the target domain undirected graph are represented as follows:

[0111] z i =∑α in v n ;

[0112] Among them, v n Let α represent the first shallow feature and the second shallow feature of the nth node in the undirected graph of the source domain and the undirected graph of the target domain, where n = 1, 2, ..., N, and N represents the number of nodes in the undirected graph of the source domain and the undirected graph of the target domain. The number of nodes in the undirected graph of the source domain and the undirected graph of the target domain is the same as the number of training data in the source domain and the target domain, respectively. in Let z represent the edge value of the edge connecting the i-th node and the n-th node in the undirected graph of the source domain and the undirected graph of the target domain. i This represents the source domain fusion feature and target domain fusion feature of the i-th node in the source domain undirected graph and the target domain undirected graph.

[0113] Figure 3 This is a schematic diagram illustrating intra-domain feature fusion and inter-domain feature alignment provided in an embodiment of the present invention. Please refer to... Figure 3 F S Represents the first shallow feature of the source domain training data, F T Let F represent the second shallow feature of the target domain training data. Taking the case where both the source and target domain training data have four data points as an example, the self-fusion unit uses the first shallow feature F based on these four source domain training data points. SWhen constructing the source domain undirected graph, the four first shallow features corresponding to the four source domain training data are used as nodes in the source domain undirected graph, i.e. Figure 3 The white hollow circle ○ in the self-fusion unit section is shown, and the query matrix Q, key matrix K and value matrix V of each node are calculated, so as to calculate the edge value of the edge connecting two nodes based on the query matrix Q, key matrix K and value matrix V.

[0114] The process of constructing an undirected graph in the target domain using self-fusion units is similar to that of constructing an undirected graph in the source domain, with the only difference being that the four second-shallow features corresponding to the four target domain training data are used as nodes in the undirected graph of the target domain. Figure 3 The gray solid circle in the self-fusion unit section ●.

[0115] After constructing the source domain undirected graph and the target domain undirected graph, intra-domain feature fusion is performed in both the target domain and the source domain. Specifically, as follows: Figure 3 As shown, the i-th node in the undirected graph of the source domain is denoted as N. i S When fusing features within a domain, node N needs to be fused according to the yellow dashed line in the self-fusion unit section. i S The corresponding first shallow feature is fused with the first shallow features corresponding to the four nodes in the source domain undirected graph, thus obtaining the source domain fused feature for each node; similarly, for the i-th node N in the target domain undirected graph... i T When fusing features within the domain, node N will be... i T The corresponding second shallow feature is fused with the second shallow features of the four nodes in the undirected graph of the target domain to obtain the target domain fused feature of each node.

[0116] Furthermore, the step of aligning inter-domain features based on the source domain fusion features and the target domain fusion features to obtain the final fusion features includes:

[0117] The source domain fusion feature of each node in the source domain undirected graph is taken as the first node, and the target domain fusion feature of each node in the target domain undirected graph is taken as the second node. The edge value of the cross edge connecting the first node and the second node is calculated to construct an undirected graph to represent the relationship between the source domain and the target domain.

[0118] For each first node and second node in the undirected graph representing the relationship between the source and target domains, the corresponding source domain fusion feature and target domain fusion feature are fused with the target domain fusion feature and source domain fusion feature of itself and the adjacent second node and first node, respectively, to obtain the final fusion feature corresponding to each first node and second node.

[0119] For details, please continue to see Figure 3 When performing inter-domain feature alignment, the source domain fused features of each node in the source domain undirected graph are taken as the first node, i.e. Figure 3 The white hollow circle ○ in the cross-fusion unit section represents the target domain fusion feature of each node in the undirected graph of the target domain as the second node. Figure 3 The gray solid circle ● in the cross-fusion unit section should be noted that in the undirected graph of the target domain or the undirected graph of the source domain, the edge connecting two adjacent nodes is called a "self-edge". When aligning features between domains, the edge connecting the first and second adjacent nodes is called a "cross-edge". When constructing an undirected graph to represent the relationship between the source and target domains, the calculation method of the cross-edge value is the same as the calculation method of the self-edge value when fusion features within the domain, so it will not be repeated here.

[0120] Please see Figure 3 The yellow dashed line in the clock cross-fusion unit section represents the first node N of the i-th node. i 1 When performing inter-domain feature alignment, the source domain fusion feature of the node is fused with its own source domain fusion feature and the target domain fusion feature of the adjacent second node to obtain the first node N. i 1 The corresponding final fusion feature; and for the i-th second node N i 2 Then, by fusing its target domain fusion features with its own target domain fusion features and the source domain fusion features of the adjacent first node, the second node N can be obtained. i 2 The corresponding final fusion features. It should be noted that "adjacent first node" and "adjacent second node" here refer to the first node connected to the second node through a cross edge and the second node connected to the first node through a cross edge, respectively.

[0121] In practice, the cross-fusion layer can include multiple self-fusion units and cross-fusion units. For example, the cross-fusion layer includes a first self-fusion unit, a first cross-fusion unit, a second self-fusion unit, a second cross-fusion unit, a third self-fusion unit, and a third cross-fusion unit connected in sequence. The final fusion features corresponding to the first node and the second node in the previous-level cross-fusion unit are input into the next-level self-fusion unit and then used as nodes of the source domain undirected graph and the target domain undirected graph in the next-level self-fusion unit, respectively. The final fusion features corresponding to the first node and the second node in the last-level cross-fusion unit are used as inputs to the fully connected layer.

[0122] The steps of inputting the final fused features into the fully connected layer to obtain the predicted probabilities of the target domain and the source domain include:

[0123] The final fusion feature corresponding to the first node is input into the fully connected layer to obtain the prediction probability of the source domain, and the final fusion feature corresponding to the second node is input into the fully connected layer to obtain the prediction probability of the target domain.

[0124] The steps include: calculating the loss value based on the predicted probability of the target domain, the predicted probability of the source domain, the ground truth value of the source domain training data, and the pseudo-label of the target domain training data; performing backpropagation based on the loss value to adjust the parameters of the land cover classification model to be trained until the preset conditions are met; and so on.

[0125] Based on the predicted probabilities of the source domain and the ground truth values ​​of the source domain training data, calculate the loss value of the source domain classification loss function; the source domain classification loss function is expressed as:

[0126]

[0127] In the formula, y nc This represents the true value of the nth source domain training data belonging to the cth class, where c = 1, 2, ..., C, and C represents the number of classes. nc This represents the predicted probability that the nth source domain training data belongs to the cth category;

[0128] Based on the predicted probability of the target domain and the pseudo-labels of the target domain training data, calculate the loss value of the target domain classification loss function; the target domain classification loss function is expressed as:

[0129]

[0130] In the formula, The pseudo-label of the nth source domain training data belongs to the cth class. This represents the predicted probability that the nth target domain training data belongs to the cth category.

[0131] The loss values ​​of the source domain classification loss function and the target domain classification loss function are weighted and summed to obtain the total loss value. Then, backpropagation is performed based on the loss value to adjust the parameters of the land cover classification model to be trained until the preset conditions are met.

[0132] As can be seen from the above embodiments, the beneficial effects of the present invention are as follows:

[0133] This invention provides an unsupervised domain-adaptive polarimetric SAR land cover classification method based on cross-domain feature fusion. Since the land cover classification model is designed for the polarimetric decomposition features of polarimetric SAR images, it can combine the first polarimetric decomposition features of the source domain samples obtained during training to perform intra-domain feature fusion of the target domain and the source domain respectively, and perform inter-domain feature alignment on the obtained target domain fusion features and source domain fusion features, thus effectively alleviating the label shift problem.

[0134] In the description of this invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0135] The use of terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples" indicates that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. In addition, those skilled in the art can combine and integrate the different embodiments or examples described in this specification.

[0136] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. An unsupervised domain-adaptive polarimetric SAR land cover classification method based on cross-domain feature fusion, characterized in that, include: The polarimetric SAR image to be processed is acquired and subjected to polarimetric coherent decomposition to obtain the polarimetric decomposition features of the polarimetric SAR image. The polarization decomposition features of the polarimetric SAR image are input into a pre-trained land cover classification model. The land cover classification model performs intra-domain feature fusion of the target domain and the source domain based on the polarization decomposition features of the polarimetric SAR image and the first polarization decomposition features of the source domain samples obtained during the training process. The obtained target domain fusion features and source domain fusion features are further aligned with inter-domain features to obtain the final fusion features. The land cover classification result is then output based on the final fusion features. The land cover classification model is trained using the following steps: Obtain a first preset number of source domain samples and a second preset number of target domain samples; After performing polarimetric coherent decomposition on the source domain samples and the target domain samples, the first polarimetric decomposition feature of each source domain sample and the second polarimetric decomposition feature of each target domain sample are obtained. The source domain samples, the target domain samples, the first polarization decomposition features, and the second polarization decomposition features are input into the land cover classification model to be trained; the land cover classification model to be trained includes: a domain balancing sampling module and a cross-domain feature fusion module, and the cross-domain feature fusion module includes: a shallow feature extractor and a cross-fusion layer; Based on the source domain samples, the target domain samples, the first polarization decomposition feature, and the second polarization decomposition feature, a domain-balanced training dataset is constructed using a domain-balanced sampling module. The domain-balanced training dataset includes a source domain training dataset and a target domain training dataset. The source domain training dataset includes source domain training data of different source domain categories, and the target domain training dataset includes target domain training data of different types of pseudo-labels. A third preset number of source domain training data and target domain training data are obtained respectively, and the first shallow feature of each source domain training data and the second shallow feature of each target domain training data are extracted using the shallow feature extractor. The first shallow features of the training data from each source domain and the second shallow features of the training data from each target domain are input into the cross-fusion layer. The final fused features are obtained through intra-domain feature fusion and inter-domain feature alignment. The final fused features are input into the fully connected layer to obtain the predicted probabilities of the target domain and the source domain. The total loss value is calculated based on the predicted probability of the target domain, the predicted probability of the source domain, the ground truth value of the source domain training data, and the pseudo label of the target domain training data. Backpropagation is then performed based on the total loss value to adjust the parameters of the land cover classification model to be trained until the preset conditions are met. The cross-fusion layer includes self-fusion units and cross-fusion units; The steps of inputting the first shallow features of the training data from each source domain and the second shallow features of the training data from each target domain into the cross-fusion layer, and obtaining the final fused features through intra-domain feature fusion and inter-domain feature alignment, include: The first shallow features of the training data from each source domain and the second shallow features of the training data from each target domain are respectively input into the self-fusion unit; The self-fusion unit generates an undirected graph of the source domain based on the first shallow features and generates an undirected graph of the target domain based on the second shallow features. Intra-domain feature fusion is performed based on the source domain undirected graph and the target domain undirected graph respectively to obtain source domain fusion features and target domain fusion features; Inter-domain feature alignment is performed based on the source domain fusion features and the target domain fusion features to obtain the final fusion features.

2. The unsupervised domain adaptive polarimetric SAR land cover classification method based on cross-domain feature fusion according to claim 1, characterized in that, The steps of constructing a domain-balanced training dataset using the domain-balanced sampling module based on the source domain samples, the target domain samples, the first polarization decomposition features, and the second polarization decomposition features include: Based on the first preset number of source domain samples, calculate the cluster center for each source domain category; For each target domain sample, calculate the Wishart distance between each pixel and the cluster center of each source domain category, and determine the pseudo-label of each pixel based on the source domain category corresponding to the smallest Wishart distance; The first polarization decomposition feature of each source domain sample is cropped into multiple first sub-blocks. After obtaining the same number of first sub-blocks under each source domain category through random sampling, each first sub-block is used as source domain training data to form the source domain training dataset. The second polarization decomposition features of each target domain sample are cropped into multiple second sub-blocks. After obtaining the same number of second sub-blocks with the same pseudo-labels through random sampling, each second sub-block is used as the target domain training data to form the target domain training dataset. A domain-balanced training dataset is formed based on the source domain training dataset and the target domain training dataset.

3. The unsupervised domain adaptive polarimetric SAR land cover classification method based on cross-domain feature fusion according to claim 1, characterized in that, The self-fusion unit generates the source domain undirected graph and the target domain undirected graph according to the following steps: Using the first shallow features of the training data from each source domain as nodes in the undirected graph of the source domain, the query matrix of each node in the undirected graph of the source domain is calculated respectively. Q Key matrix K Sum matrix V ; Based on the query matrix of each node in the undirected graph of the source domain Q Key matrix K Sum matrix V Calculate the edge value of the edge connecting two adjacent nodes in the source domain undirected graph to generate the source domain undirected graph; Using the second shallow features of the training data for each target domain as nodes in the undirected graph of the target domain, the query matrix of each node in the undirected graph of the target domain is calculated respectively. Q Key matrix K Sum matrix V ; Based on the query matrix of each node in the undirected graph of the target domain Q Key matrix K Sum matrix V Calculate the edge value of the edge connecting two adjacent nodes in the undirected graph of the target domain, and generate the undirected graph of the target domain.

4. The unsupervised domain adaptive polarimetric SAR land cover classification method based on cross-domain feature fusion according to claim 3, characterized in that, The source domain undirected graph\the target domain undirected graph Query matrix of nodes Q Key matrix K Sum matrix V Represented as: ； in, , , Represent the i-th generation in the source domain undirected graph and the j-th generation in the target domain undirected graph, respectively. Query matrix of nodes Q Key matrix K Sum matrix V， , , This represents the preset weight matrix. , , Indicates the preset bias; In an undirected graph of the source domain and an undirected graph of the destination domain, the connection of the first... The node and the first The edge values ​​of the edges of each node are represented as follows: ； in, This indicates the transpose operation. This represents the Softmax activation function. In the undirected graph of the source domain and the undirected graph of the destination domain, the connection of the first... The node and the first The edge values ​​of the edges of each node. .

5. The unsupervised domain adaptive polarimetric SAR land cover classification method based on cross-domain feature fusion according to claim 4, characterized in that, The steps of fusing intra-domain features based on the source domain undirected graph and the target domain undirected graph respectively to obtain source domain fused features and target domain fused features include: For each node in the source domain undirected graph, its corresponding first shallow feature is fused with the first shallow features of all nodes in the source domain undirected graph to obtain the source domain fused feature of that node. For each node in the target domain undirected graph, its corresponding second shallow feature is fused with the second shallow features corresponding to all nodes in the target domain undirected graph to obtain the target domain fused feature of that node. The source domain undirected graph\the target domain undirected graph The source domain fusion feature and target domain fusion feature of each node are represented as follows: ； in, Represents the first undirected graph in the source domain and the second undirected graph in the destination domain. The first shallow feature and the second shallow feature of each node. , This represents the number of nodes in the source domain undirected graph and the target domain undirected graph, and the number of nodes in the source domain undirected graph and the target domain undirected graph is the same as the number of nodes in the source domain training data and the target domain training data. In the undirected graph of the source domain and the undirected graph of the destination domain, the connection of the first... The node and the first The edge values ​​of the edges of each node. Represents the first undirected graph in the source domain and the second undirected graph in the destination domain. Source domain fusion characteristics and target domain fusion characteristics of each node.

6. The unsupervised domain adaptive polarimetric SAR land cover classification method based on cross-domain feature fusion according to claim 5, characterized in that, The steps for aligning inter-domain features based on source domain fusion features and target domain fusion features to obtain the final fusion features include: The source domain fusion feature of each node in the source domain undirected graph is taken as the first node, and the target domain fusion feature of each node in the target domain undirected graph is taken as the second node. The edge value of the cross edge connecting the first node and the second node is calculated to construct an undirected graph to represent the relationship between the source domain and the target domain. For each first node and second node in the undirected graph representing the relationship between the source domain and the target domain, the corresponding source domain fusion feature and target domain fusion feature are respectively fused with the target domain fusion feature and source domain fusion feature of itself and the adjacent second node and first node to obtain the final fusion feature corresponding to each first node and second node.

7. The unsupervised domain adaptive polarimetric SAR land cover classification method based on cross-domain feature fusion according to claim 6, characterized in that, The steps of inputting the final fused features into the fully connected layer to obtain the predicted probabilities of the target domain and the source domain include: The final fusion feature corresponding to the first node is input into the fully connected layer to obtain the prediction probability of the source domain, and the final fusion feature corresponding to the second node is input into the fully connected layer to obtain the prediction probability of the target domain. The steps include: calculating a loss value based on the predicted probability of the target domain, the predicted probability of the source domain, the ground truth value of the source domain training data, and the pseudo-label of the target domain training data; performing backpropagation based on the loss value to adjust the parameters of the land cover classification model to be trained until preset conditions are met; and so on. Calculate the loss value of the source domain classification loss function based on the predicted probability of the source domain and the true value of the source domain training data; Calculate the loss value of the target domain classification loss function based on the predicted probability of the target domain and the pseudo-labels of the target domain training data; The loss values ​​of the source domain classification loss function and the target domain classification loss function are weighted and summed to obtain the total loss value. Then, backpropagation is performed based on the loss value to adjust the parameters of the land cover classification model to be trained until the preset conditions are met.

8. The unsupervised domain adaptive polarimetric SAR land cover classification method based on cross-domain feature fusion according to claim 7, characterized in that, The source domain classification loss function is expressed as follows: ； In the formula, Indicates the first The training data from the source domain belong to the first... The true values ​​of each category, , Indicates the number of categories. Indicates the first The training data from the source domain belong to the first... The predicted probabilities of each category; The target domain classification loss function is expressed as follows: In the formula, No. The training data from the source domain belong to the first... Pseudo-labels for each category, Indicates the first The training data for the target domain belongs to the first... The predicted probability of each category.

Images