Feature unwrapping convolutional neural network transformation model domain adaptive method

Abstract

The invention provides a novel feature decoupling domain adaptive transformation model method. The method comprises the following steps of: projecting sample features of a source domain and a target domain into a space by utilizing a feature mapping thought, decomposing the projection features into domain invariant features and domain specific features by utilizing a feature gradient distribution mask, and gradually decoupling in a loss function iteration process; and finally, an attention mechanism in DETR is optimized to adapt to a variable domain transfer process, and the model is forced to carry out accurate feature alignment on decomposed domain invariant features and domain specific features, so that a downstream target detection task is further realized. The algorithm is subjected to experimental verification between a heterogenous SAR image and a heterogenous modal remote sensing image, the performance of the algorithm is superior to that of an SOTA method, and the effectiveness of the algorithm is proved by each module.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing target detection, and in particular to a feature disentangled convolutional neural network transformation model domain adaptation method. Background Art

[0002] Synthetic Aperture Radar (SAR) is a high-resolution radar imaging system. Compared with traditional optical remote sensing systems, SAR has the ability to detect targets in various environments. Even in bad weather conditions, such as at night or under clouds, it can detect and image and obtain corresponding high-resolution microwave images. SAR's Automatic Target Recognition (ATR) technology aims to detect and accurately classify targets of interest by parsing SAR images. The goal of target detection is to automatically detect and locate the position and size of the target of interest in a SAR image containing multiple targets, accurately locate the target and provide the target's bounding box, thereby providing an accurate target area for subsequent classification tasks.

[0003] Traditional SAR target detection algorithms are usually based on shallow learning and require manually designed features and classifiers for classification. In contrast, SAR target detection algorithms based on deep learning automatically learn features and classifiers from training data through end-to-end training under the guidance of detection and recognition results. However, deep learning is a data-driven algorithm, and the parameters of the deep model are not set manually, but are usually learned from a large amount of labeled training data. Therefore, when both the training set and the test set come from SAR data with the same joint probability distribution, most of the existing deep learning-based target detection and recognition methods have achieved good results. However, most of the existing deep learning-based research results do not directly solve the pain points of SAR image target detection and recognition tasks, that is, these research results still have problems with inconsistent source data distribution and generalization of the algorithm itself.

[0004] In fact, in actual engineering applications, it is very common that the distribution of training data and test data is inconsistent. For example, images generated by modeling and simulation are used in training, while measured images are used in testing; for another example, SAR images obtained by one sensor are used as training sets, while SAR images obtained by another sensor are used as test sets. This distribution difference between heterogeneous data will cause the model obtained on the training set to be difficult to obtain the expected results on the test set. Even if the training sets are similar but there are large differences in the feature level, the generalization performance and learning reusability of the model will be greatly reduced. The above situation can be initially solved by the domain adaptation method in transfer learning, that is, the training set and test set with inconsistent distribution are input as the source domain and target domain respectively into the constructed domain adaptation network structure, and the appropriate feature alignment distribution is sought, so as to map the data of the target domain to the source domain, and then realize the downstream target detection and recognition tasks. The core idea of ​​domain adaptation is to align the feature distribution of the source domain and the target domain, realize the feature transfer between domains, and thus solve the problem of inconsistent distribution of heterogeneous SAR image samples. In the continuous research on this problem, it is found that by extracting and decomposing domain-invariant features from the features of a specific domain, the performance of the algorithm in domain transfer and regression bounding box determination can be improved. In addition, a vision transformation model is designed to perform in-depth analysis of domain-specific knowledge to further improve detection accuracy.

[0005] Due to the special imaging mechanism of SAR, whether the domain adaptation between heterogeneous data can be successfully transferred depends mainly on how much information (domain-invariant features) is shared between the two domains. Although some domain adaptation methods do find domain-invariant features through the idea of ​​domain adversarial learning, they are basically limited to the learning of explicit domain-invariant features, while ignoring the information knowledge implicit in the specific domain features, which are the so-called implicit domain-invariant features. These domain adaptation methods mainly focus on obtaining domain-invariant features and require the simultaneous training of shared feature extractors and classifiers for each specific domain. In addition, ignoring domain-specific features will also lead to the model being unable to adapt at an extremely high level, because this information may help improve generalization performance, especially when the source domain samples have high complexity.

[0006] Incorporating SAR image data and incorporating edge data features from adjacent source domains significantly increases the complexity of data alignment between the source domain and the target domain, which not only increases the complexity of extracting domain-invariant features, but also has an adverse effect on the accuracy of data transmission. Domain adaptation methods do not consider the utility of domain-invariant features and domain-specific features, but simply use the extracted features for adversarial learning. Although this relatively rough feature alignment method can achieve domain transfer to a certain extent, it is limited by the characteristics of the heterogeneous image itself. Therefore, when facing highly sensitive images such as SAR, the extension of this method will cause the model performance to remain at a low-precision level of only low-level feature alignment.

[0007] In the process of gradually improving the accuracy of target detection in heterogeneous SAR images, the following two issues should also be considered. The first issue is how to design a more effective feature decoupling method in combination with the model architecture to adapt to more accurately separate domain-invariant features and domain-specific features, and make each feature respond effectively in the domain adaptation process. Another issue is that a more complete and adaptive model needs to be designed to solve the domain transfer gap between heterogeneous SAR images or remote sensing images of different modalities. The detection accuracy of SAR image targets can be improved through convolutional neural networks (CNN). Summary of the invention

[0008] The purpose of the present invention is to further improve the target detection accuracy of heterogeneous SAR images and propose a feature disentangled convolutional neural network transformation model domain adaptation method, which includes the following steps:

[0009] Step 1: SAR images acquired by different imaging platforms are defined as labeled source domains and unlabeled target domains;

[0010] Step 2: Extract its features through CNN and then map them to the latent space, separate the domain-invariant part from the domain-specific part according to the channel dimension; divide the projected features into the domain-invariant part and the domain-specific part through the feature gradient distribution mask;

[0011] Step 3: Adopt the feature disentanglement strategy to completely decouple the separated domain-invariant features and domain-specific features and make them independent of each other;

[0012] Step 4: Adopt the Detection Transformer (DETR) approach to update the multiple attention mechanisms of the variable domain adaptive transformer model to handle the process of matching the domain-invariant features with the domain-specific features obtained in the previous step.

[0013] Further, the step 1 is specifically as follows:

[0014] Defining the source domain of labels for heterogeneous SAR images acquired by different imaging platforms and the unlabeled target domain in, is the source domain image, is the bounding box label, is the class label, N s is the total number of source domain images, N t is the total number of target domain images, i is the i-th image;

[0015] In addition to N s image In addition, there are two labels corresponding to the image in the source domain, namely: bounding box label and class labels Similarly, the target domain is unlabeled, so it only contains N t images

[0016] Further, the step 2 is specifically as follows:

[0017] To achieve independent feature decomposition, the module consists of three parts: feature inverse mapping, feature gradient distribution mask and independent cross-domain features.

[0018] 1) Inverse feature mapping: For heterogeneous SAR images, the multi-channel cross-domain features after the feature extraction encoder have domain invariance and domain specificity. Invertible Neural Networks (INN) are used as the feature mapping of the latent space. After obtaining the domain-invariant channels in the latent space, they are returned to the original feature space through its reversible process.

[0019] This feature map is defined as the function G f , and its inverse process is defined as

[0020] 2) Feature gradient distribution mask: Decompose the features according to the feature channel dimension, and use the gradient of the classification head relative to the i-th average integrated channel feature as the evaluation indicator, which is defined as:

[0021]

[0022] Among them, c x and Represents the output of the classification head of the image sample x and the i-th average mixed channel feature, respectively. Where C, H, and W represent the number of channels, height, and width, respectively, so the gradient after average pooling for each channel is In addition, different samples of the same category have different gradient distributions on the river channel, so the union of all samples is considered and the domain-invariant channel is selected for each category;

[0023] Creating a Feature Distribution Library As the channel mask, where J is the category number, M b All components in are initialized to 1. For a sample x belonging to class J, the mask of class J and the i-th channel is defined as:

[0024]

[0025] Among them, τ is the threshold of feature gradient change, so the mask Used to update the feature distribution library M b ,as follows:

[0026]

[0027] Where, ⊙ is the Hadamard product. When all masks of each source sample in the class are also set to 1, M b The component only retains the value of 1; on the contrary, in any other scenario, it is assigned a value of 0. The threshold parameter τ has a decisive influence on the feature decomposition. When τ is set too small, it will result in retaining a large number of channels, thereby limiting the denoising effect for specific domain features; on the contrary, if τ becomes too large, there is a risk of ignoring many domain-invariant channels; after processing each source sample, it is possible to derive a specialized M for each category. b , in order to isolate the semantic domain invariant channel, directly access this M b Allows the method to perform selections without recalculating gradients;

[0028] 3) Disentanglement of Feature Correlations

[0029] Since the target domain samples are completely unlabeled SAR images, the spherical network clustering method is first used to obtain the pseudo-labels of the target domain samples. The Jth cluster center of the target domain is denoted as O J , only one subset is marked as a confident sample in, Specified by the nearest cluster center, d0 is a constant set to 0.05; the feature distance d(·,·) here is measured by cosine similarity. In addition, if the number of samples that meet the conditions in a category is less than 2, the category will not be sampled in the current traversal;

[0030] Then, the extracted source and confident target features f s and f t Mapping to h via a reversible feature map s and h t , the feature distribution library M b Allow h s and h t Decompose into and in and consists of channels with mask value 1, since it is assumed that domain-invariant features should follow the same distribution, if we swap and To get two features h' s and h' t , their distributions should also be the same, and this process can be written as:

[0031]

[0032] Among them, ⊕ represents the connection operation, and in addition, the inversion feature f' s ,f' t Should be respectively with h' s ,h' t With the same distribution, in order to train a reversible feature map, the definition of contrast domain difference loss is introduced as follows:

[0033]

[0034] in, It is a function of measuring distribution differences. The introduction of this loss reduces the intra-class distance of SAR image features and increases the inter-class distance.

[0035] In order to further ensure that the domain invariant components of the source and target samples follow the same distribution, the regularization component loss is introduced Defined as

[0036]

[0037] According to the description of formula (5), formula (6) makes the restriction and It becomes possible to make the latent space more compact.

[0038] Furthermore, in step 2, the optimal τ is set to select C / 2 channels to represent the domain invariant component.

[0039] Further, the step three is specifically as follows:

[0040] Feature alignment and regression positioning are performed on the DETR framework, so that the domain-invariant features and domain-specific features obtained above are represented as a standard feature token, where the category embedding Bounding Box Embedding u and v represent the two corresponding image axes, c is estimated along these two image axes, and j is the feature dimension;

[0041] In order to align the features of the attention mechanism in the transformation model framework, [[c di ,c ds ],[b di ,b ds ]] represents the feature h s and h t , since it is certain whether the heterogeneous SAR image comes from the source domain or the target domain, in order to facilitate calculation, the source domain s is specified as The target domain t is specified as

[0042] The encoder integrates remote visual information into each image annotation, and the decoder decodes the objects according to the context of the image to output the prediction results; for the image-level feature tokens, content embeddings, and position embeddings of the query token sequence, they are written in the following forms respectively as W = [w di , w ds , C = [c di , c ds and B = [b di , b ds ;

[0043] Relying on the variable attention mechanism, the domain-invariant content residual part △c di is calculated as follows:

[0044]

[0045] where is the attention matrix between n queries and n' < N sampling points; Φ di is a subset of the image value Ξ of the embedding , offset by Ξ from the query position, and are both learnable linear projection matrices;

[0046] Similarly, the domain-specific content residual part △c ds is estimated as:

[0047]

[0048] where Φ ds is a subset of the image value embedding , and are both learnable linear projection matrices, and Φ ds is sampled at n' < N positions with the same offset ξ as the query position in Equation (7);

[0049] GRL operates as an identity function during forward computation and reverses the gradient during backpropagation, which enables it to deceive the domain discriminator, thereby reducing the domain difference in c di , which is the key to solving domain shift;

[0050] The domain-specific attention matrix is defined as follows:

[0051] Ψ ds = ModelAttention[GRL(c di + b di ), (c ds + b ds )] (9)

[0052] Among them, ModelAttention is the model attention mechanism, from c di Extract specific domain features into c ds This, in turn, helps to learn c more efficiently by making it domain invariant di ;

[0053] For the object-level feature tags, content, and position embeddings of the query token sequence, they are respectively in the following form: Q = [q di ,q ds ], and Since the goal of object-level feature alignment is to capture object relations in object-query sequences, it does not use strained attention but full attention;

[0054] Referring to the form of formula (7), the domain-invariant self-attention residual feature is written as

[0055]

[0056] in, is a standard self-attention mechanism in softmax form, is the projection value embedding of domain invariant features, is the projection matrix, and similarly recursively, the domain invariant feature is directly used here The generated attention matrix is ​​used to process the two branches, thus encouraging feature exchange based on the similarity of object-level domain-invariant features;

[0057] Correspondingly, the domain-specific self-attention residual feature is expressed as:

[0058]

[0059] The same calculation method as formula (8) is used to calculate the embedded specific domain value from the entire object-level feature: in, Represents projection matrix, domain invariant object-level features The domain gap of is reduced by gradient inversion operation;

[0060] The detection transformation model treats target detection as an integrated prediction problem and uses a bipartite matching technique to establish a one-to-one correspondence between the model output and the ground truth target or background category in the SAR image; due to the lack of semantic labels on the target domain, target detectors are prone to inaccurate matches between target queries and real objects on the target domain; to solve this problem, the outputs of different decoder layers are aggregated and the output of each decoder layer is constrained during training to produce consistent bidirectional matches. The corresponding loss function is written as:

[0061]

[0062] Among them, l de is the number of decoder layers, is the reference output obtained by averaging the predictions of all decoder layers, is the set of corresponding category embeddings and bounding box embeddings, It is a consistency loss, which is used to measure the bidirectional matching consistency between two predictions; specifically, it is a combined JS divergence (Jensen-Shannon-divergenc, JSD) function classification output and the distance regression output between the L1 bounding box, such as:

[0063]

[0064] in, The total number of query object levels, JS divergence function and λ1 are balanced hyper-parameters set to 0.3, which makes the outputs of different decoder layers consistent and improves the distinguishability of DETR in the target domain.

[0065] Further, the step 4 is specifically as follows:

[0066] The feature disentanglement domain adaptive transformation model method for target detection in heterogeneous SAR images includes the following objective functions:

[0067]

[0068] in is the detection loss of DETR, and α, β, and θ are the hyperparameters that weigh the corresponding loss functions.

[0069] The beneficial effect of the present invention is that a new feature decoupled domain adaptive transformation model method is proposed. The method first uses the idea of ​​feature mapping to project the sample features of the source domain and the target domain into space, and then uses the feature gradient distribution mask to decompose the projected features into domain-invariant features and domain-specific features, and gradually decouples them in the process of iterative loss function. Finally, the attention mechanism in DETR is optimized to adapt to the variable domain transfer process, forcing the model to accurately align the decomposed domain-invariant features and domain-specific features, thereby further realizing downstream target detection tasks. The algorithm was experimentally verified between heterogeneous SAR images and heterogeneous modal remote sensing images, and its performance is better than the existing state-of-the-art (SOTA) method, and each module has proved its effectiveness. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 This is a network framework diagram for heterogeneous SAR image detection in the present invention;

[0071] Figure 2 are typical scenes from two heterogeneous remote sensing image ship datasets, where Figure 2 (a) and Figure 2 (b) from the GF-3 dataset, Figure 2 (c) and Figure 2 (d) from SSDD;

[0072] Figure 3 A typical scene from two heterogeneous SAR image aircraft datasets, where Figure 3 (a) and Figure 3 (b) from the MAR20 dataset, Figure 3 (c) and Figure 3 (d) from SAR aircraft;

[0073] Figure 4 is the feature distribution before and after the domain adaptation method proposed by t-SNE, where Figure 4 (a) is the functional distribution of GF-3 and SSDD, Figure 4 (b) is the feature distribution of GF-3 and SSDD after domain adaptation. Figure 4 (c) is the characteristic distribution of MAR20 and SAR aircraft, Figure 4 (d) shows the feature distribution of MAR20 and SAR aircraft after domain adaptation;

[0074] Figure 5 It is the feature alignment heatmap with and without feature disentanglement in the GF-3→SSDD task, where Figure 5 (a) is the first scene without feature separation. Figure 5 (b) is the first feature separation scene. Figure 5 (c) is the second scene without feature separation. Figure 5 (d) is the second scene feature separation;

[0075] Figure 6 The comparative experimental results of target detection in heterogeneous SAR images for the GF-3→SSDD task are shown in Figure 2. Figure 6 (a1) is GF-3FRCNN, Figure 6 (b1) is GF-3CSTD, Figure 6 (c1) is GF-3FDBM, Figure 6 (d1) is GF-3SAHSC, Figure 6 (e1) is GF-3DETR, Figure 6 (f1) is GF-3RST, Figure 6 (g1) is GF-3DENO, Figure 6 (h1) is GF-3DSKT, Figure 6 (i1) is the method recommended by GF-3, Figure 6 (j1) is the GF-3 ground truth, Figure 6 (a2) is SSDD FRCNN, Figure 6 (b2) is SSDD CSTD, Figure 6 (c2) is SSDD FDBM, Figure 6 (d2) is SSDD SAHSC, Figure 6 (e2) is SSDD DETR, Figure 6 (f2) is SSDD RST, Figure 6 (g2) is SSDD DENO, Figure 6 (h2) is SSDD DSKT, Figure 6 (i2) is the method recommended by SSDD, Figure 6 (j2) is the SSDD ground truth, where the solid lines are correct detections, the dashed lines are wrong detections, and the dotted lines are missed detections;

[0076] Figure 7 The comparative experimental results of SSDD→GF-3 task on non-uniform SAR image target detection are shown in Figure 2. Figure 7 (a1) is SSDD FRCNN, Figure 7 (b1) is SSDD CSTD, Figure 7 (c1) is SSDD FDBM, Figure 7 (d1) is SSDD SAHSC, Figure 7 (e1) is SSDD DETR, Figure 7 (f1) is SSDD RST, Figure 7 (g1) is SSDD DENO, Figure 7 (h1) is SSDD DSKT, Figure 7 (i1) is the method recommended by SSDD, Figure 7 (j1) is the SSDD ground truth, Figure 7 (a2) is GF-3FRCNN, Figure 7 (b2) is GF-3CSTD, Figure 7 (c2) is GF-3FDBM, Figure 7 (d2) is GF-3SAHSC, Figure 7 (e2) is GF-3DETR, Figure 7 (f2) is GF-3RST, Figure 7 (g2) is GF-3DENO, Figure 7 (h2) is GF-3DSKT, Figure 7 (i2) is the method recommended by GF-3, Figure 7 (j2) is the GF-3 ground truth, where the solid line is the correct detection, the dashed line is the wrong detection, and the dotted line is the missed detection;

[0077] Figure 8 The comparative experimental results of heterogeneous remote sensing image target detection in the MAR20→SAR aircraft mission are shown in Figure 2. Figure 8 (a1) is MAR20 FRCNN, Figure 8 (b1) is MAR20 CSTD, Figure 8 (c1) is MAR20 FDBM, Figure 8 (d1) is MAR20SAHSC, Figure 8 (e1) is MAR20 DETR, Figure 8 (f1) is MAR20 RST, Figure 8 (g1) is MAR20 DENO, Figure 8 (h1) is MAR20 DSKT, Figure 8 (i1) is the method recommended by MAR20, Figure 8 (j1) is the MAR20 ground truth, Figure 8 (a2) is the SAR aircraft FRCNN, Figure 8 (b2) is the SAR aircraft CSTD, Figure 8 (c2) is the SAR aircraft FDBM, Figure 8 (d2) is the SAR aircraft SAHSC, Figure 8 (e2) is the SAR aircraft DETR, Figure 8 (f2) is the SAR aircraft RST, Figure 8 (g2) is SAR aircraft DENO, Figure 8 (h2) is the SAR aircraft DSKT, Figure 8 (i2) The method proposed for SAR aircraft, Figure 8 (j2) is the SAR aircraft ground truth, the solid line is the correct detection, the dashed line is the wrong detection, and the dotted line is the missed detection.

[0078] Fig. 9 The comparative experimental results of heterogeneous remote sensing image target detection in the SAR aircraft → MAR20 mission are shown in Figure 2. Fig. 9 (a1) is the SAR aircraft FRCNN, Fig. 9 (b1) is the SAR aircraft CSTD, Fig. 9 (c1) is the SAR aircraft FDBM, Fig. 9 (d1) is the SAR aircraft SAHSC, Fig. 9 (e1) is the SAR aircraft DETR, Fig. 9 (f1) is the SAR aircraft RST, Fig. 9 (g1) is the SAR aircraft DENO, Fig. 9 (h1) is the SAR aircraft DSKT, Fig. 9 (i1) The method proposed for SAR aircraft, Fig. 9 (j1) is the SAR aircraft ground truth, Fig. 9 (a2) is MAR20 FRCNN, Fig. 9 (b2) is MAR20 CSTD, Fig. 9 (c2) is MAR20 FDBM, Fig. 9 (d2) is MAR20 SAHSC, Fig. 9 (e2) is MAR20 DETR, Fig. 9 (f2) is MAR20RST, Fig. 9 (g2) is MAR20 DENO, Fig. 9 (h2) is MAR20 DSKT, Fig. 9 (i2) is the method recommended by MAR20, Fig. 9 (j2) is the MAR20 ground truth, where the solid lines represent correct detections, the dashed lines represent incorrect detections, and the dotted lines represent missed detections. DETAILED DESCRIPTION

[0079] The present invention will be further described below in conjunction with the accompanying drawings and examples of the present invention.

[0080] A feature disentangled convolutional neural network transformation model domain adaptation method, comprising the following steps:

[0081] Step 1: SAR images acquired by different imaging platforms are defined as labeled source domains and unlabeled target domains;

[0082] Step 2: Extract its features through CNN and then map them to the latent space, separate the domain-invariant part from the domain-specific part according to the channel dimension; divide the projected features into the domain-invariant part and the domain-specific part through the feature gradient distribution mask;

[0083] Step 3: Adopt the feature disentanglement strategy to completely decouple the separated domain-invariant features and domain-specific features and make them independent of each other;

[0084] Step 4: Adopt the detection transformation model approach to update the multiple attention mechanism of the variable domain adaptive transformation model, which handles the process of matching the domain-invariant features with the domain-specific features obtained in the previous step.

[0085] Further, the step 1 is specifically as follows:

[0086] Defining the source domain of labels for heterogeneous SAR images acquired by different imaging platforms and the unlabeled target domain in, is the source domain image, is the bounding box label, is the class label, N s is the total number of source domain images, N t is the total number of target domain images, i is the i-th image;

[0087] In addition to N s image In addition, there are two labels corresponding to the image in the source domain, namely: bounding box label and class labels Similarly, the target domain is unlabeled, so it only contains N t images

[0088] This method is based on the baseline network and mainly consists of two parts, such as Figure 1 The overall domain adaptation SAR image target detection framework is shown. The main idea of ​​the present invention is to design an effective feature extraction and decomposition method to completely decouple domain-invariant features from domain-specific features and effectively perform downstream feature alignment and target regression and positioning tasks.

[0089] Further, the step 2 is specifically as follows:

[0090] To achieve independent feature decomposition, the module consists of three parts: feature inverse mapping, feature gradient distribution mask and independent cross-domain features.

[0091] 1) Inverse feature mapping: For heterogeneous SAR images, the multi-channel cross-domain features after the feature extraction encoder have domain invariance and domain specificity. Invertible Neural Networks (INN) are used as the feature mapping of the latent space. After obtaining the domain-invariant channels in the latent space, they are returned to the original feature space through its reversible process.

[0092] This feature map is defined as the function G f , and its inverse process is defined as

[0093] 2) Feature gradient distribution mask: Decompose the features according to the feature channel dimension, and use the gradient of the classification head relative to the i-th average integrated channel feature as the evaluation indicator, which is defined as:

[0094]

[0095] Among them, c x and Represents the output of the classification head of the image sample x and the i-th average mixed channel feature, respectively. Where C, H, and W represent the number of channels, height, and width, respectively, so the gradient after average pooling for each channel is In addition, different samples of the same category have different gradient distributions on the river channel, therefore, the union of all samples is considered and the domain-invariant channel is selected for each category;

[0096] Creating a Feature Distribution Library As the channel mask, where J is the category number, M b All components in are initialized to 1. For a sample x belonging to class J, the mask of class J and the i-th channel is defined as:

[0097]

[0098] Among them, τ is the threshold of feature gradient change, so the mask Used to update the feature distribution library M b ,as follows:

[0099]

[0100] Where, ⊙ is the Hadamard product. When all masks of each source sample in the class are also set to 1, M b The component only retains the value of 1; on the contrary, in any other scenario, it is assigned the value of 0. The threshold parameter τ has a decisive influence on the feature decomposition. When τ is set too small, it will result in a large number of channels being retained, thereby limiting the denoising effect on specific domain features; on the contrary, if τ becomes too large, there is a risk of ignoring many domain-invariant channels; the optimal setting τ ensures that C / 2 channels are accurately selected to represent the domain-invariant component. After processing each source sample, a specialized M can be derived for each category. b , in order to isolate the semantic domain invariant channel, directly access this M b Allows the method to perform selections without recalculating gradients;

[0101] 3) Disentanglement of Feature Correlations

[0102] Since the target domain samples are completely unlabeled SAR images, the spherical network clustering method is first used to obtain the pseudo-labels of the target domain samples. The Jth cluster center of the target domain is denoted as O J , only one subset is marked as a confident sample in, Specified by the nearest cluster center, d0 is a constant set to 0.05; the feature distance d(·,·) here is measured by cosine similarity. In addition, if the number of samples that meet the conditions in a category is less than 2, the category will not be sampled in the current traversal;

[0103] Then, the extracted source and confident target features f s and f t Mapping to h via a reversible feature map s and h t , the feature distribution library M b Allow h s and ht Decompose into and in and consists of channels with mask value 1, since it is assumed that domain-invariant features should follow the same distribution, if we swap and To get two features h' s and h' t , their distributions should also be the same, and this process can be written as:

[0104]

[0105] Among them, ⊕ represents the connection operation, and in addition, the inversion feature f' s ,f' t Should be respectively with h' s ,h' t With the same distribution, in order to train a reversible feature map, the definition of contrast domain difference loss is introduced as follows:

[0106]

[0107] in, It is a function of measuring distribution differences. The introduction of this loss reduces the intra-class distance of SAR image features and increases the inter-class distance.

[0108] In order to further ensure that the domain invariant components of the source and target samples follow the same distribution, the regularization component loss is introduced Defined as

[0109]

[0110] According to the description of formula (5), formula (6) makes the restriction and It becomes possible to make the latent space more compact.

[0111] Further, the step three is specifically as follows:

[0112] Feature alignment and regression positioning are performed on the DETR framework, so that the domain-invariant features and domain-specific features obtained above are represented as a standard feature token, where the category embedding Bounding Box Embedding u and v represent the two corresponding image axes, c is estimated along these two image axes, and j is the feature dimension;

[0113] In order to align the features of the attention mechanism in the transformation model framework, [[c di ,c ds ],[b di ,bds represents the feature h in the form of s and h t , since it is determined whether the heterologous SAR image is from the source domain or the target domain, for the convenience of calculation, the source domain s is specified as {-1} j , and the target domain t is specified as

[0114] The encoder integrates remote visual information into each image annotation, and the decoder decodes the object according to the context of the image to output the prediction result; for the image-level feature markers, content embeddings, and position embeddings of the query token sequence, they are written in the following forms respectively: W = [w di , w ds , C = [c di , c ds and B = [b di , d ds ;

[0115] Relying on the variable attention mechanism, the domain-invariant content residual part △c di is calculated as follows:

[0116]

[0117] where is the attention matrix between n queries and n' < N sampling points; Φ di is a subset of the image value Ξ of the embedding , offset by Ξ from the query position, and are both learnable linear projection matrices;

[0118] Similarly, the domain-specific content residual part △c ds is estimated as:

[0119]

[0120] where Φ ds is a subset of the image value embedding , and are both learnable linear projection matrices, and Φ ds is sampled at n' < N positions with the same offset ξ as the query position in Equation (7);

[0121] GRL operates as an identity function during forward calculation and reverses the gradient during backpropagation, which enables it to "fool" the domain discriminator, thereby reducing the domain difference in c di , which is the key to solving domain shift;

[0122] The domain-specific attention matrix is ​​defined as follows:

[0123] Ψ ds =ModelAttention[GRL(c di +b di ),(c ds +b ds )] (9)

[0124] Among them, ModelAttention is the model attention mechanism, from c di Extract specific domain features into c ds This, in turn, helps to learn c more efficiently by making it domain invariant di ;

[0125] For the object-level feature tags, content, and position embeddings of the query token sequence, they are respectively in the following form: Q = [q di ,q ds ], and Since the goal of object-level feature alignment is to capture object relations in object-query sequences, it does not use strained attention but full attention;

[0126] Referring to the form of formula (7), the domain-invariant self-attention residual feature is written as

[0127]

[0128] in, is a standard self-attention mechanism in softmax form, is the projection value embedding of domain invariant features, is the projection matrix, and similarly recursively, the domain invariant feature is directly used here The generated attention matrix is ​​used to process the two branches, thus encouraging feature exchange based on the similarity of object-level domain-invariant features;

[0129] Correspondingly, the domain-specific self-attention residual feature is expressed as:

[0130]

[0131] The same calculation method as formula (8) is used to calculate the embedded specific domain value from the entire object-level feature: in, Represents projection matrix, domain invariant object-level features The domain gap of is reduced by gradient inversion operation;

[0132] The detection transformation model treats target detection as an integrated prediction problem and uses a bipartite matching technique to establish a one-to-one correspondence between the model output and the ground truth target or background category in the SAR image; due to the lack of semantic labels on the target domain, target detectors are prone to inaccurate matches between target queries and real objects on the target domain; to solve this problem, the outputs of different decoder layers are aggregated and the output of each decoder layer is constrained during training to produce consistent bidirectional matches. The corresponding loss function is written as:

[0133]

[0134] Among them, l de is the number of decoder layers, is the reference output obtained by averaging the predictions of all decoder layers, is the set of corresponding category embeddings and bounding box embeddings, is a consistency loss that measures the bidirectional matching consistency between two predictions; specifically, it is a combined JS divergence (Jensen-Shannon-divergenc) classification output and the distance regression output between the L1 bounding boxes, such as:

[0135]

[0136] in, The total number of query object levels, JS divergence function and λ1 are balanced hyper-parameters set to 0.3, which makes the outputs of different decoder layers consistent and improves the distinguishability of DETR in the target domain.

[0137] Further, the step 4 is specifically as follows:

[0138] The feature disentanglement domain adaptive transformation model method for target detection in heterogeneous SAR images includes the following objective functions:

[0139]

[0140] in is the detection loss of DETR, and α, β, and θ are the hyperparameters that weigh the corresponding loss functions.

[0141] Experimental results and analysis

[0142] In order to verify the effectiveness of the proposed method, a series of comparative tests were conducted with the SOTA method while focusing on the ablation analysis of the components of the proposed framework. This comprehensive experimental analysis not only verifies the superiority of the proposed method, but also clarifies the performance improvement brought by each specific detail, thus laying a solid foundation for its application and further refinement.

[0143] A. Dataset and Experimental Setup

[0144] Firstly, the different distribution datasets of two heterogeneous spaceborne SAR imaging platforms, GF-3 dataset and SAR Ship Detection Dataset (SSDD), are used as source domain and target domain respectively to verify the proposed domain adaptive target detection method. The GF-3 dataset is collected and processed by the team, and its images come from China's first C-band multi-polarization spaceborne SAR with a resolution of up to 1m. Realizing spatiotemporal coordination, all-weather and all-day earth observation targets is an important foundation for the "Gaofen" project. After the "No. 3" constellation network is put into operation, the maximum revisit period of global observation will be shortened from 84 hours to 12 hours, and the average revisit period will be shortened from 12 hours to 4.8 hours, which will comprehensively improve the business application level of China's multi-polarization SAR satellite data, further meet the urgent needs of users in a wide range of industries such as ocean, disaster prevention, water conservancy, aerospace, and meteorology, and expand and apply microwave remote sensing data in high temporal resolution.

[0145] SSDD is the first open dataset widely used to study SOTA of deep learning-based ship detection SAR images. It is composed of three different spaceborne SAR imaging results: TerraSAR-X, Sentinel-1, and RadarSat-2. According to the survey, as many as 46.59% of the public reports choose SSDD to study deep learning-based SAR ship detection. Undoubtedly, this situation reveals the popularity and great influence of SSDD in the remote sensing community.

[0146] The corresponding imaging platform has appeared many times in existing studies and will not be repeated here.

[0147] Empirically, SAR images in both datasets are cropped from large scenes to fit the network input size of SAR images. Figure 2 The following are typical SAR image scenes from two datasets. Figure 2 It can be seen that the SAR images of the GF-3 dataset are more complex and have higher image quality than those of the SSDD dataset. In addition, the specific technical details of the two datasets are shown in Table 1.

[0148] Table 1 Comparison of technical parameters between GF-3 and SSDD datasets

[0149]

[0150] At the same time, the above two heterogeneous SAR image target detection datasets have been proven to have domain shift at the feature level, which will not be repeated here. Without losing the generality, in order to fully verify the effectiveness of the proposed method, in subsequent experiments, the present invention uses the GF-3 dataset and SSDD as the source domain and target domain respectively, and defines two domain adaptive SAR image target detection tasks: GF-3→SSDD and SSDD→GF-3.

[0151] In order to fully verify the performance of the proposed algorithm, the present invention also introduces the problem of the difficulty of detecting aircraft targets from heterogeneous SAR images, and considers the ability of the proposed method to achieve domain transfer between optical remote sensing and SAR images of different modalities. The detection of aircraft targets is much more complicated than that of ships at sea, not only because of the complexity of the airport environment and the interference of noise. At the same time, as far as the target itself is concerned, the extraction of deep semantic features of aircraft is relatively difficult, and this difficulty is more exposed in the process of domain transfer between multi-platform images.

[0152] The MAR20 dataset collects 3,842 high-resolution remote sensing images of 60 airports in the United States, Russia and other countries through Google Earth, and most of the image sizes are 800 × 800 pixels. MAR20 uses the open source image annotation tool "Labelme" to annotate a total of 22,341 object instances, and provides horizontal bounding boxes and oriented bounding boxes for each instance.

[0153] Wang et al. constructed an open dataset of large-scale complex scene SAR aircraft targets, named SAR-AIRcraft. The SAR-AIRcraft image resolution is 1m, contains 4368 aircraft pieces, involving 7 fine-grained aircraft types, and has the characteristics of scene complexity, category richness, target density, noise interference, task diversity, and multi-scale. Considering the scale of the airport and the number of parked aircraft, the dataset mainly selected image data from three civil airports, namely Shanghai Hongqiao Airport, Beijing Capital Airport and Taiwan Taoyuan Airport. All images in the SAR-AIRcraft dataset are from the Gaofen-3 satellite, using single polarization, 1m spatial resolution, and cluster beam imaging mode.

[0154] Figure 3 For optical remote sensing aircraft target scenes and SAR aircraft target scenes. It can be intuitively seen that there is a large direct difference between the two, and it can be inferred that the difference at the feature level is even greater. Accordingly, in order to fully verify the effectiveness of the method of the present invention, in subsequent experiments, the MAR20 dataset and SAR-AIRcraft are used as the source domain and target domain respectively, and two domain adaptive SAR image target detection tasks, MAR20→SAR-AIRcraft and SAR-AIRcraft→MAR20, are defined.

[0155] In addition, both aircraft datasets are fine-grained recognition of aircraft objects, but the specific categories of aircraft objects are not discussed in the experiment of the present invention, so the present invention uniformly replaces the category information in the label with "AIRcraft". At the same time, for the purpose of experiment and for unification, the label format of all the above datasets is changed to PASCALVOC 2007 format. Similarly, the figure shows two different scenes from the two datasets respectively, and the differences between heterogeneous images of different modality imaging platforms can also be more intuitively displayed.

[0156] In order to quantitatively demonstrate the performance of the proposed target detection method, the present invention uses the following three common metrics for target detection tasks to evaluate all methods. The three parameter indicators are True Positives (TP), False Positives (FP), True Negatives (FN) and False Negatives (FN), which are predicted by the model as correct positive samples, false positive samples, correct negative samples and false negative samples, respectively. In order to quantitatively evaluate the experiment, the present invention uses the following three validation indicators precision (PR), recall (RE) and mean PR (mAP) to measure the performance of the method:

[0157]

[0158] PR represents the proportion of accurately predicted instances among all predictions made by the model, while RE represents the share of real-world instances correctly identified and predicted by the model. mAP represents the area under the Precision-Recall curve, which effectively characterizes the comprehensive efficacy of the proposed detection technique.

[0159] For all experiments of this method, the present invention uses a momentum of 0.9 and a weight decay of 1×10 -4Stochastic gradient descent (SGD) is used. Based on experience, the three hyperparameters α, β and θ proposed in equation (14) are set to 0.8, 0.6 and 0.2 in the following comparative experiments. In the analytical experiment section, the sensitivity of these three hyperparameters will be analyzed in more detail. The intersection over union (IoU) is a key indicator for evaluating target detection performance. Its threshold is set to 0.5, which is consistent with the method adopted in almost all contemporary SAR image target detection methods. The learning rates of the backbone network and the transformation model are set to 1e-5 and 1e-4, respectively, and the batch size is set to 16. The learning rate decay is set to 0.1 and applied after 40 elements of the 50 traversal training plan. All experiments are performed on a GIGABYTE X299WU8 workstation with Intel Core(R) i910920X CPU, Nvidia TESLAV100 Turbo GPU and 32G RAM under Centos 7. The code of the present invention is built on the backbone network of the DETR baseline network ResNet-101 based on the Pytorch framework.

[0160] B. Ablation Studies and Related Analysis

[0161] In-depth effectiveness experiments are conducted on the proposed domain adaptive transformation model. The effectiveness of the method is verified through a series of ablation experiments and analysis experiments.

[0162] 1) Ablation study of feature disentanglement: In order to more accurately extract and separate invariant features and domain-specific features, thereby promoting domain adaptation, three steps are taken in Section III-B. The present invention establishes two important objective functions, namely the contrast domain difference loss function and the regularization component loss function. Below, ablation experiments will be used to verify how the target detection performance of heterogeneous SAR images changes with or without (w / o) these two loss functions. Without loss of generality, the two domain adaptive target detection tasks GF-3→SSDD and MAR20→SAR-AIRcraft are selected as examples to conduct related experiments on different types of target detection.

[0163] From Table 2, we can first see that the proposed feature disentanglement module is crucial to improving the detection performance, which directly improves the detection rate by 19.1% and 17.2%. This also shows that if only feature alignment is considered, there are obvious limitations in the research on domain adaptive object detection. Secondly, the two loss functions proposed in the module also contribute significantly to improving the detection performance. Their presence increases the detection rate by 11.8% and 3.4% respectively compared to their absence (e.g., GF-3→SSDD task). However, in comparison, l cdd It is superior to l in improving target detection performance r This is also due to lcdd The function of is to better separate domain-invariant features from domain-specific features, which can be directly used for subsequent feature alignment and regression positioning.

[0164] Table 2. Hybrid experiment results of heterogeneous image object detection (%)

[0165]

[0166] 2) Ablation study of feature alignment: Feature alignment is a necessary step to eliminate domain offset, so this article will not verify whether it has an impact on the following aspects. Similarly, this paper only conducts ablation experiments on the objective function proposed in this module to verify its impact on the effectiveness of the algorithm. Specifically, taking GF-3→SSDD and MAR20→SAR-AIRcraft as examples, we discuss w / ol cm The target detection rate of heterogeneous SAR images is relatively good. It can be clearly seen from Table 3 that the loss function does improve the detection performance, verifying the effectiveness of the proposed module.

[0167] Table 3 Explosion experiment of heterogeneous image objects (%) W / O l cm

[0168]

[0169] Please also note that in formula (13), JS divergence is used to describe the distance of category output. JS divergence is a special form of divergence, and other special forms include α-divergence, F-divergence, etc. Here, ablation experiments are also performed on these different forms of divergence to verify whether JS divergence is selected to achieve the best detection performance.

[0170] Obviously, it can be seen from Table 4 that the use of JS-divergence can output the highest target detection rate. Since the JS-divergence algorithm solves the problem of asymmetric feature samples in the K-divergence algorithm, and the category features of heterogeneous remote sensing images are asymmetric, the use of the JS-divergence algorithm can obtain the best detection performance.

[0171] Table 4 Detection results of explosion experiments on heterogeneous image objects in different forms (%)

[0172]

[0173] 3) Backbone network analysis: In the above experiments, ResNet-101 was used as the backbone network, and ResNet also has backbone networks with other parameters. Therefore, in this ablation experiment, the present invention selected three backbone networks, ResNet-34, ResNet-50 and ResNet-152, as comparisons to verify the experimental settings used in the present invention. As can be seen from Table 5, ResNet-101 has higher target detection accuracy than ResNet-34 and ResNet-50. Although the detection accuracy of ResNet-152 is 0.1% higher than that of ResNet-101, it has more iterative parameters, which will increase the network burden too much during the calculation process, and the gradient explosion phenomenon is prone to occur. Therefore, considering all factors, using ResNet-101 as the backbone network is the best cost-effective choice.

[0174] Table 5 Detection of different backbone networks in the explosion experiment of heterogeneous image objects (%)

[0175]

[0176] 4) Visualization Analysis,The first thing to consider in visualization analysis is the domain transfer effect of feature distribution. From the perspective of domain adaptation, the domain gap between heterogeneous SAR images lies in the different distribution of target features. Figure 4 The t-Distributed Stochastic Neighbor Embedding (t-SNE) used for feature dimensionality reduction reduces the target features of heterogeneous SAR images from multiple dimensions to two dimensions, thereby observing the ability of domain adaptation to change feature distribution to eliminate domain gaps. Although it is unknown what features are specifically represented by the dimensionality reduction of feature distribution, the domain gap between source domain samples and target domain samples with large differences in feature distribution is indeed reduced in dimension by the domain adaptation method proposed in the present invention, which undoubtedly verifies the effectiveness of the method of the present invention.

[0177] In addition, we visualize and compare the feature alignment heatmaps with and without feature disentanglement operation, which is an example of the GF-3→SSDD task. Figure 5 The results shown clearly show that the feature alignment process after the feature decoupling operation can pay more attention to the ship target itself, while being well separated from the nearshore background, which is more conducive to improving the fitting ability of the model.

[0178] C. Comparative experimental results analysis

[0179] In addition to the ablation experiment to verify the effectiveness of the proposed method, further verification of its effectiveness in the same research domain also requires advantages over existing SOTA methods. Therefore, 6 domain adaptation SOTA methods are used to compare with the proposed method, including 3 CNN-based methods and 3 transformation model-based methods. At the same time, the proposed method is verified to effectively suppress the domain gap using target detection benchmark methods such as CNN and transformation models. For ease of writing, the methods listed here for comparison include cross-scene target detection method (CSTD), feature decomposition-based method (FDBM) and source-based hierarchical semantic calibration method (SAHSC), all of which are CNN-based domain adaptation methods; as well as Remote Sensing Teacher (RST), DenoDet (DENO) and A Domain Specific Knowledge Transformer (DSKT) domain adaptation methods based on transformation models. Accordingly, FRCNN and DETR are two benchmark methods suitable for researchers who develop new algorithms to focus on target detection.

[0180] Table 6 first shows two SAR image ship target detection tasks GF-3→SSDD and SSDD→GF-3. The present invention lists three CNN-based domain adaptation SOTA methods and three transformation model-based domain adaptation SOTA methods for comparison. These three methods have been developed by researchers in the past three years for the transmission of heterogeneous SAR or remote sensing data. In addition, the comparison results also show the baseline methods FRCNN and DETR of these two methods. Both methods only consider the source data for training and the target data for testing.

[0181] From the quantitative results in Table 6, it can be seen that the method using the domain adaptation strategy outperforms the two baseline networks in test performance, which also intuitively shows that domain adaptation can effectively eliminate the gap between domains and find suitable feature distributions from the labeled source domain and the unlabeled target domain. Although it has not been proven that the domain adaptation method based on the transformation model is always better than the domain adaptation method based on CNN, the experimental results show that the method proposed in the present invention does achieve better detection performance. From the experimental results, the detection performance of this method on the domain adaptation task is 3.6% and 5.1% higher than the suboptimal method, respectively, and significantly better than the DETR baseline network by more than 20%. It can be seen that the listed SOTA methods all use the ResNet backbone network with only slight differences in layer parameters. In addition, a large number of research papers have used ResNet as the backbone, indicating that it is suitable for domain adaptation tasks. In addition, due to the significant differences between the GF-3 dataset and SSDD in scene complexity, target feature characteristics, and noise interference, the detection performance of the two tasks is also different.

[0182] Table 6 Image ship target detection results of various methods (%)

[0183]

[0184] In addition, the present invention selects two different scenarios from the two tasks to demonstrate the results of the qualitative comparison experiment. Figure 6 and Figure 7 It can be seen that when faced with the detection task of heterogeneous SAR image targets in different scenarios, other methods are more or less prone to acquisition errors and omissions, while the method of the present invention can ensure correct detection results that are close to the true value.

[0185] Without loss of generality, the present invention also conducts detection experiments on aircraft target images of different imaging platforms, and the comparison results are shown in Table 7. In terms of overall performance, the target detection task on aircraft images is more challenging than the ship target detection task. The results show that domain adaptation from ship images to aircraft images reduces the accuracy by about 20%. This is also because it is more difficult to extract low-altitude features, such as contour features, from aircraft targets than from ship targets, and the airport background is more complex and disturbed than the sea surface. Obviously, the comparative experiments show that the method still maintains the highest detection rate in dual-domain adaptive object detection tasks. Therefore, in both types of cross-modal remote sensing image target detection tasks, it surpasses the SOTA method and achieves the best detection performance.

[0186] Table 7 Image target detection results of various methods (%)

[0187]

[0188] The present invention also selects two different scenarios from the two tasks to demonstrate the results of the qualitative comparison experiment. Figure 8 It can be seen that the domain transfer from optical remote sensing to heterogeneous modalities such as SAR is indeed very difficult, especially when dealing with small targets such as airplanes and complex backgrounds such as airports. In addition, when the model learns good target features from labeled SAR image data, the model does not work well when applied to optical remote sensing images because the background and target are sometimes mixed together and SAR images do not learn effective color features. This also makes Fig. 9 The detection accuracy of this task is not very good. However, among the SOTA methods demonstrated qualitatively, the method of the present invention is still the best method, which fully demonstrates the superiority of the method of the present invention.

[0189] The beneficial effect of the present invention lies in: a new feature decoupled domain adaptive transformation model method is proposed. This method first uses the idea of ​​feature mapping to project the sample features of the source domain and the target domain into space, and then uses the feature gradient distribution mask to decompose the projected features into domain-invariant features and domain-specific features, and gradually decouples them in the process of iterative loss function. Finally, the attention mechanism in DETR is optimized to adapt to the variable domain transfer process, forcing the model to accurately align the decomposed domain-invariant features and domain-specific features, thereby further realizing downstream target detection tasks. The algorithm was experimentally verified between heterogeneous SAR images and heterogeneous modal remote sensing images, and its performance was better than the SOTA method, and each module proved its effectiveness.

Claims

1. A feature disentangled convolutional neural network transformation model domain adaptation method, characterized by: The following steps are involved: Step 1: SAR images acquired by different imaging platforms are defined as labeled source domains and unlabeled target domains; Step 2: Extract its features through convolutional neural network and then map them to the latent space, separate the domain-invariant part from the domain-specific part according to the channel dimension, and divide the projected features into domain-invariant part and domain-specific part through feature gradient distribution mask; Step 3: Adopt the feature disentanglement strategy to completely decouple the separated domain-invariant features and domain-specific features and make them independent of each other; Step 4: Adopt the detection transformation model approach to update the multiple attention mechanism of the variable domain adaptive transformation model, which handles the process of matching the domain-invariant features with the domain-specific features obtained in the previous step.

2. The feature disentangled convolutional neural network transformation model domain adaptation method according to claim 1, characterized in that: The step 1 is specifically as follows: Defining the source domain of labels for heterogeneous SAR images acquired by different imaging platforms and the unlabeled target domain in, is the source domain image, is the bounding box label, is the class label, N s is the total number of source domain images, N t is the total number of target domain images, i is the i-th image; In addition to N s image In addition, there are two labels corresponding to the image in the source domain, namely: bounding box label and class labels Similarly, the target domain is unlabeled, so it only contains N t images 3. The feature disentangled convolutional neural network transformation model domain adaptation method according to claim 1, characterized in that: The step 2 is specifically as follows: To achieve independent feature decomposition, the module consists of three parts: feature inverse mapping, feature gradient distribution mask and independent cross-domain features. 1) Inverse feature mapping: For heterogeneous SAR images, the multi-channel cross-domain features after the feature extraction encoder have domain invariance and domain specificity. A reversible neural network is used as the feature mapping of the latent space, so that after obtaining the domain invariant channels in the latent space, it returns to the original feature space through its reversible process; This feature map is defined as the function G f , and its inverse process is defined as 2) Feature gradient distribution mask: Decompose the features according to the feature channel dimension, and use the gradient of the classification head relative to the i-th average integrated channel feature as the evaluation indicator, which is defined as: Among them, c x and Represents the output of the classification head of the image sample x and the i-th average mixed channel feature, respectively. Where C, H, and W represent the number of channels, height, and width, respectively, so the gradient after average pooling for each channel is In addition, different samples of the same category have different gradient distributions on the river channel, so the union of all samples is considered and the domain-invariant channel is selected for each category; Creating a Feature Distribution Library As the channel mask, J is the category number, M b All components in are initialized to 1. For a sample x belonging to class J, the mask of class J and the i-th channel is defined as: Among them, τ is the threshold of feature gradient change, so the mask Used to update the feature distribution library M b ,as follows: Where, ⊙ is the Hadamard product. When all masks of each source sample in the class are also set to 1, M b The component only retains the value of 1; on the contrary, in any other scenario, it is assigned a value of 0. The threshold parameter τ has a decisive influence on the feature decomposition. When τ is set too small, it will result in retaining a large number of channels, thereby limiting the denoising effect for specific domain features; on the contrary, if τ becomes too large, there is a risk of ignoring many domain-invariant channels; after processing each source sample, it is possible to derive a specialized M for each category. b , in order to isolate the semantic domain invariant channel, directly access this M b Allows the method to perform selections without recalculating gradients; 3) Disentanglement of Feature Correlations Since the target domain samples are completely unlabeled SAR images, the spherical network clustering method is first used to obtain the pseudo-labels of the target domain samples. The Jth cluster center of the target domain is denoted as O J , only one subset is marked as a confident sample in, Specified by the nearest cluster center, d0 is a constant set to 0.05; the feature distance d(·,·) here is measured by cosine similarity. In addition, if the number of samples that meet the conditions in a category is less than 2, the category will not be sampled in the current traversal; Then, the extracted source and confident target features f s and f t Mapping to h via a reversible feature map s and h t , the feature distribution library M b Allow h s and h t Decompose into and in and consists of channels with mask value 1, since it is assumed that domain-invariant features should follow the same distribution, if we swap and To get two features h' s and h' t , their distributions should also be the same, and this process can be written as: Among them, ⊕ represents the connection operation, and in addition, the inversion feature f' s ,f' t Should be respectively with h' s ,h' t With the same distribution, in order to train a reversible feature map, the definition of contrast domain difference loss is introduced as follows: in, It is a function of measuring distribution differences. The introduction of this loss reduces the intra-class distance of SAR image features and increases the inter-class distance. In order to further ensure that the domain invariant components of the source and target samples follow the same distribution, the regularization component loss is introduced Defined as According to the description of formula (5), formula (6) makes the restriction and It becomes possible to make the latent space more compact.

4. The feature disentangled convolutional neural network transformation model domain adaptation method according to claim 3, characterized in that: The step 2 is specifically as follows: in the feature gradient distribution mask 2), the optimal τ is set to select the C / 2 channel to represent the domain invariant component.

5. The feature disentangled convolutional neural network transformation model domain adaptation method according to claim 1, characterized in that: The step three is specifically as follows: Feature alignment and regression positioning are performed on the DETR framework, so that the domain-invariant features and domain-specific features obtained above are represented as a standard feature token, where the category embedding Bounding Box Embedding u and v represent the two corresponding image axes, c is estimated along these two image axes, and j is the feature dimension; In order to align the features of the attention mechanism in the transformation model framework, [[c di ,c ds ],[b di ,b ds ]] represents the feature h s and h t , since it is certain whether the heterogeneous SAR image comes from the source domain or the target domain, in order to facilitate calculation, the source domain s is specified as The target domain t is specified as The encoder integrates long-range visual information into each image annotation, and the decoder decodes the object according to the context of the image to output the prediction result; for the image-level feature labeling, content embedding and position embedding of the query label sequence, they are written as follows W = [w di ,w ds ],C=[c di ,c ds ] and B = [b di ,b ds ]; Relying on the variable attention mechanism, the domain-invariant content residual △c di The calculation is as follows: Among them, is the attention matrix between n queries and n' < N sampling points; Φ di is the embedding is a subset of the image values Ξ of, offset from the query position by Ξ, and are both learnable linear projection matrices; Similarly, the domain-specific content residual △c ds Estimated to be: where, Φ ds is a subset of the image value embedding , and are both learnable linear projection matrices, and Φ ds is sampled at n' < N positions with the same offset ξ as the query position in Equation (7); GRL operates as an identity function during the forward computation and inverts the gradient during backpropagation, which enables it to fool the domain discriminator and thus reduce c di The domain differences in the , which is the key to solving the domain transfer; The domain-specific attention matrix is ​​defined as follows: P ds =ModelAttention[GRL(c di +b di ),(c ds +b ds )] (9) Among them, ModelAttention is the model attention mechanism, from c di Extract specific domain features into c ds This, in turn, helps to learn c more efficiently by making it domain invariant di ; For the object-level feature tags, content, and position embeddings of the query token sequence, they are respectively in the following form: Q = [q di ,q ds ], and Since the goal of object-level feature alignment is to capture object relations in object-query sequences, it does not use strained attention but full attention; Referring to the form of formula (7), the domain-invariant self-attention residual feature is written as in, is a standard self-attention mechanism in softmax form, is the projection value embedding of domain invariant features, is the projection matrix, and similarly recursively, the domain invariant feature is directly used here The generated attention matrix is ​​used to process the two branches, thus encouraging feature exchange based on the similarity of object-level domain-invariant features; Correspondingly, the domain-specific self-attention residual feature is expressed as: The same calculation method as formula (8) is used to calculate the embedded specific domain value from the entire object-level feature: in, Represents projection matrix, domain invariant object-level features The domain gap of is reduced by gradient inversion operation; The detection transformation model treats target detection as an integrated prediction problem and uses a bipartite matching technique to establish a one-to-one correspondence between the model output and the ground truth target or background category in the SAR image; due to the lack of semantic labels on the target domain, target detectors are prone to inaccurate matches between target queries and real objects on the target domain; to solve this problem, the outputs of different decoder layers are aggregated and the output of each decoder layer is constrained during training to produce consistent bidirectional matches. The corresponding loss function is written as: Among them, l de is the number of decoder layers, is the reference output obtained by averaging the predictions of all decoder layers, is the set of corresponding category embeddings and bounding box embeddings, is the consistency loss, which is used to measure the bidirectional matching consistency between two predictions; specifically, it is a combined JS divergence function classification output and the distance regression output between the L1 bounding boxes, such as: in, The total number of query object levels, JS divergence function and λ1 are balanced hyperparameters set to 0.3, which makes the outputs of different decoder layers consistent and improves the discriminability of DETR in the target domain.

6. The feature disentangled convolutional neural network transformation model domain adaptation method according to claim 1, characterized in that: The step 4 is specifically as follows: The feature disentanglement domain adaptive transformation model method for target detection in heterogeneous SAR images includes the following objective functions: in is the detection loss of DETR, and α, β, and θ are the hyperparameters that weigh the corresponding loss functions.

Images