UDA SAR ATR method and device based on multi-modal feature fusion and global local domain alignment

Abstract

The invention discloses a UDA SAR ATR method and device based on multi-modal feature fusion and global local domain alignment. The method comprises the steps that a to-be-recognized actually-measured full-aperture SAR image is acquired; dividing the image into a group of sub-aperture images; constructing a scattering topological graph of the image; processing a group of sub-aperture images and a scattering topological graph of the image by adopting a trained deep recognition network to obtain the category of a target contained in the image; the trained deep recognition network is obtained by training based on a preset data set; the preset data set comprises a plurality of categories of simulation full-aperture SAR images and actual measurement full-aperture SAR images, each simulation full-aperture SAR image has a real domain label and a category label, each actual measurement full-aperture SAR image only has a real domain label, the domain label represents whether the image is a simulation image or an actual measurement image, and the category label represents whether the image is an actual measurement image. The category tag represents a category of a target in the image. According to the method, the cross-domain SAR target recognition performance can be improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of radar image processing, and in particular relates to a UDA SAR ATR method and device for multi-modal feature fusion and global-local domain alignment. Background Art

[0002] Synthetic aperture radar (SAR) uses the mode of actively emitting microwaves and has the ability to image all day and all weather. Therefore, SAR has become an important device for earth observation and has been widely used in military defense, environmental monitoring, disaster warning and urban planning. In recent years, automatic target recognition (ATR) has gradually become one of the most important research topics in the field of SAR. In the past decade, deep learning has achieved great success in the field of computer vision. Therefore, more and more scholars have introduced deep learning into SAR ATR and proposed a variety of recognition models with excellent performance. As a typical data-driven method, deep learning-based models usually require a large amount of data to effectively capture the intricate features and patterns in the data. However, compared with optical images, SAR images are more difficult to acquire and annotate, which poses a challenge to the application of deep learning in SAR ATR. Given the special imaging mechanism of SAR, generating simulated SAR images through computer-aided design (CAD) modeling and electromagnetic calculation is a feasible way to alleviate data shortage. However, due to the approximate and simplified processing in the electromagnetic simulation and model design process, there is a non-negligible distribution difference between simulated and measured SAR images, which makes the recognition model trained on simulated images not well generalized to measured images. Therefore, how to bridge the distribution difference between simulated images and measured images and accurately identify measured images based on simulated images has become an engineering problem with practical application value. To address this problem, there are currently some solutions based on the theory of unsupervised domain adaptation (UDA), but the cross-domain recognition performance of these methods is still not good enough, resulting in inaccurate recognition results for the categories of targets in measured full-aperture SAR images. Summary of the invention

[0003] In order to solve the above problems existing in the prior art, the present invention provides a UDA SAR ATR method and device for multi-modal feature fusion and global-local domain alignment.

[0004] The technical problem to be solved by the present invention is achieved through the following technical solutions:

[0005] The present invention provides a UDA SAR ATR method for multimodal feature fusion and global-local domain alignment, comprising:

[0006] Acquire the measured full-aperture SAR image to be identified;

[0007] Using a sub-aperture decomposition algorithm, the measured full-aperture SAR image is divided into a group of sub-aperture images; the group of sub-aperture images includes a plurality of different sub-aperture images;

[0008] Constructing a scattering topology map of the measured full-aperture SAR image;

[0009] Using a trained deep recognition network, the group of sub-aperture images and the scattering topology map of the measured full-aperture SAR image are processed to obtain the category of the target contained in the measured full-aperture SAR image;

[0010] The trained deep recognition network is obtained by training an initial deep recognition network based on a preset data set; the preset data set contains multiple categories of simulated full-aperture SAR images and multiple categories of measured full-aperture SAR images, and each simulated full-aperture SAR image in the preset data set has a real domain label and a real category label, and each measured full-aperture SAR image has a real domain label, the domain label is used to indicate whether the image is a simulated image or a measured image, and the category label is used to indicate the category of the target in the image.

[0011] The present invention also provides a UDA SAR ATR device for multimodal feature fusion and global local domain alignment, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; the memory is used to store a computer program; and the processor is used to implement the steps of the above-mentioned UDA SAR ATR method for multimodal feature fusion and global local domain alignment when executing the program stored in the memory.

[0012] Compared with the prior art, the present invention has the following beneficial effects:

[0013] The present invention adopts a sub-aperture decomposition algorithm to divide a measured full-aperture SAR image into a plurality of different sub-aperture images, and constructs a scattering topology map of the measured full-aperture SAR image at the same time. Afterwards, a trained deep recognition network is used to process the sub-aperture images and the scattering topology map to obtain the category of the target contained in the measured full-aperture SAR image. The present invention introduces the sub-aperture information of SAR images in traditional visual feature extraction, and significantly enhances the ability of visual features to identify targets through sub-aperture feature fusion; the present invention introduces the position information of key points through position encoding when constructing the scattering topology map of the target in the SAR image, and further combines it with the descriptor information of the key points to better model the topological structure of the target, thereby significantly improving the ability of scattering topological features to identify targets; the present invention introduces a gating mechanism when fusing multimodal features, and adaptively mines complementary information between different modal features and suppresses conflicting information through the gating feature fusion module, thereby significantly improving the fusion effect; the present invention introduces the idea of ​​gradient-based difficult sample mining when aligning the global distribution of simulated images and measured images, and further corrects the weight of difficult samples through power transformation and Softmax normalization with temperature, thereby reducing the negative impact of difficult samples on subsequent local domain alignment, and significantly improving the cross-domain recognition performance of the model. In short, the present invention significantly improves the cross-domain recognition performance through multimodal feature fusion and global local domain alignment, thereby improving the recognition accuracy of target categories in measured full-aperture SAR images.

[0014] The present invention will be further described in detail below with reference to the accompanying drawings and specific implementation methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a flowchart of a UDA SARATR method for multimodal feature fusion and global-local domain alignment provided by an embodiment of the present invention;

[0016] Figure 2 is a schematic diagram of a sub-aperture decomposition process provided by an embodiment of the present invention;

[0017] Figure 3 is a schematic diagram of the structure of a sub-aperture feature fusion network provided by an embodiment of the present invention;

[0018] Figure 4 is a schematic diagram of the structure of a position-aware graph attention network provided by an embodiment of the present invention;

[0019] Figure 5 is a schematic diagram of the structure of a gated feature fusion module provided in an embodiment of the present invention;

[0020] Figure 6 is a schematic diagram of the structure of an identifier provided by an embodiment of the present invention;

[0021] Figure 7 It is a schematic diagram of the structure and forward propagation of the deep recognition network provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0022] The present invention is further described in detail below with reference to specific embodiments, but the embodiments of the present invention are not limited thereto.

[0023] Figure 1 is a flow chart of a UDA SARATR method for multimodal feature fusion and global-local domain alignment provided by an embodiment of the present invention, such as Figure 1 As shown, the method includes:

[0024] S101: Acquire a measured full-aperture SAR image to be identified.

[0025] Here, the measured full-aperture SAR image to be identified may be one or more images, which is not limited in the present invention. The processing steps in S102 to S104 below are steps for processing each measured full-aperture SAR image.

[0026] S102, using a sub-aperture decomposition algorithm to divide the measured full-aperture SAR image into a group of sub-aperture images; a group of sub-aperture images includes a plurality of different sub-aperture images.

[0027] S103, constructing a scattering topology map of the measured full-aperture SAR image.

[0028] S104. Using a trained deep recognition network, a group of sub-aperture images and a scattering topology map of the measured full-aperture SAR image are processed to obtain the category of the target contained in the measured full-aperture SAR image; wherein the trained deep recognition network is obtained by training an initial deep recognition network based on a preset data set; the preset data set contains multiple categories of simulated full-aperture SAR images and multiple categories of measured full-aperture SAR images, and each simulated full-aperture SAR image in the preset data set has a real domain label and a real category label, and each measured full-aperture SAR image has only one real domain label, the domain label is used to indicate whether the image is a simulated image or a measured image, and the category label is used to indicate the category of the target in the image.

[0029] In some embodiments, the above S102 is implemented by the following steps:

[0030] S1021. Perform a one-dimensional Fourier transform on the measured full-aperture SAR image along the azimuth direction to convert the measured full-aperture SAR image into a range-Doppler domain to obtain a Doppler domain image.

[0031] S1022, performing a zero removal operation and a window removal operation on the Doppler domain image in sequence to obtain a window removal image.

[0032] S1023, dividing the spectrum of the windowed image into three parts at equal intervals along the azimuth direction to obtain three sub-spectra, wherein the three sub-spectra have overlapping parts, and each sub-spectra contains 1 / 2 of the original spectrum of the windowed image.

[0033] S1024. Perform windowing, zero-filling, one-dimensional inverse Fourier transform, quarter power magnitude (QPM) normalization and center cropping operations on each sub-spectrum in sequence to obtain a sub-aperture image of a preset size; the three sub-aperture images corresponding to the three sub-spectra constitute a group of sub-aperture images of the measured full-aperture SAR image.

[0034] For example, Figure 2 As shown, a full-aperture SAR image X is divided into a set of sub-aperture images X sub1 ,X sub2 ,X sub3 The steps are as follows:

[0035] (1a) performing a one-dimensional Fourier transform on the full-aperture SAR image along the azimuth direction, converting the image into the range-Doppler domain, and obtaining a Doppler domain image;

[0036] (1b) removing the zero elements added to the spectrum of the Doppler domain image for image smoothing to obtain a zero-free image;

[0037] (1c) Multiplying the frequency spectrum of the zero-removed image by the inverse of a preset window function to remove the influence of the window function and obtain a window-removed image; illustratively, the preset window function is a Taylor window function with a peak sidelobe ratio of -35 dB.

[0038] (1d) Dividing the spectrum of the windowed image into three equal parts along the azimuth direction to obtain three sub-spectra, wherein the three sub-spectra overlap with each other, and each sub-spectra contains 1 / 2 of the original spectrum of the windowed image;

[0039] (1e) performing a windowing operation on each sub-spectrum independently. For example, the window function used is a Taylor window with a peak-to-sidelobe ratio of -35 dB to suppress side lobes, thereby obtaining a windowed sub-spectrum;

[0040] (1f) performing a zero-filling operation on the windowed sub-spectrum, adding sampling points to smooth the image, and obtaining a zero-filled sub-spectrum;

[0041] (1g) performing a one-dimensional inverse Fourier transform on the zero-filled sub-spectrum, transforming the spectrum into the time domain, and obtaining a sub-aperture image, which is a complex-valued sub-aperture image;

[0042] (1h) Each sub-aperture image is independently normalized to one-quarter power amplitude to improve its contrast;

[0043] Specifically, the specific steps of normalizing the quarter power amplitude are as follows:

[0044] (1h1) taking a modulus of the sub-aperture image to obtain an amplitude sub-aperture image;

[0045] (1h2) assigning the amplitude of each pixel point of the amplitude sub-aperture image whose amplitude is greater than 1 to 1 to obtain a truncated image;

[0046] (1h3) performing a power transformation of a preset power on the truncated image to improve its contrast; illustratively, the preset power is 0.5;

[0047] (1h4) Use maximum and minimum normalization to normalize the pixel amplitude range of the power-transformed amplitude sub-aperture image to [0, 1];

[0048] (1i) Each sub-aperture image after the contrast is improved is independently center-cropped and cropped to a preset size to obtain a final set of sub-aperture images X sub1 ,X sub2 ,X sub3 ; For example, the preset size is 64×64 pixels.

[0049] In some embodiments, the “constructing a scattering topology map of the measured full-aperture SAR image” in S103 is implemented by the following steps:

[0050] S1031. Perform contrast enhancement processing on the measured full-aperture SAR image to obtain a processed image.

[0051] Exemplarily, the measured full-aperture SAR image may be normalized to one-quarter power amplitude to enhance the contrast of the measured full-aperture SAR image and obtain a processed image.

[0052] S1032, extracting scattering topological points of the processed image, and generating a descriptor for each extracted scattering topological point; each extracted scattering topological point has a two-dimensional space coordinate, and the two-dimensional space coordinate includes: a range coordinate and an azimuth coordinate.

[0053] Exemplarily, the SAR-SIFT algorithm can be used to extract scattering topological points of the processed image, and each scattering topological point contains three attributes: a descriptor r that describes its local backscattering characteristics in the form of a gradient, a range coordinate p, and an azimuth coordinate q.

[0054] S1033 . For each scattering topological point, generate an initial feature of the scattering topological point according to a two-dimensional spatial coordinate and a descriptor of the scattering topological point.

[0055] The above S1033 is implemented by the following steps:

[0056] A1. For each scattering topology point in the scattering topology map of the measured full-aperture SAR image, respectively calculate the K-bit position code of the range coordinate and the K-bit position code of the azimuth coordinate of the scattering topology point, where K is a preset integer.

[0057] Here, the calculation formula of the k-th code (k-th element) in the K-bit position code of the distance coordinate of a scattering topological point or the k-th code in the K-bit position code of the azimuth coordinate of the scattering topological point is: Among them, the value of k is 1, 2, ..., K, even represents an even number, odd represents an odd number, and pos represents the distance coordinate or azimuth coordinate of the scattering topological point.

[0058] A2. Concatenate the K-bit position codes of the distance coordinates and the K-bit position codes of the azimuth coordinates to obtain the position code of the two-dimensional space coordinates of the scattering topological point.

[0059] Exemplarily, the expression of the position encoding e of the two-dimensional spatial coordinates of a scattering topological point is: e=PE(p)||PE(q), PE(p) represents the K-bit position encoding of the distance coordinate of the scattering topological point, PE(q) represents the K-bit position encoding of the azimuth coordinate of the scattering topological point, and || represents splicing.

[0060] A3. Calculate the average value of the descriptor of the scattering topological point and the position encoding of the two-dimensional space coordinates of the scattering topological point, and use the average value as the initial feature of the scattering topological point.

[0061] Exemplarily, the expression of the initial characteristic v of a scattering topological point is: v=(e+r) / 2.

[0062] The present invention uses both the position information and descriptor information of key points in the extraction of scattering topological features based on the SAR-SIFT algorithm, and maps the two-dimensional position information to a high-dimensional space through position coding theory, so that it can be effectively combined with the descriptor information, significantly improving the ability of the scattering topological features to identify targets.

[0063] S1034, taking each scattering topological point as a node, taking the initial features of the scattering topological point as the initial features of the node, and determining multiple neighboring nodes of each node according to the Euclidean distance between the nodes, generating a unidirectional edge from each node to each neighboring node of the node, and obtaining a node set of the measured full-aperture SAR image. The scattering topology graph composed of the edge set ε

[0064] Here, the Euclidean distance between any two nodes can be calculated using two-dimensional space coordinates. For example, the calculation formula of the Euclidean distance between any two nodes is: Among them, the i'th node and the j'th node are two arbitrary nodes, p i' is the distance coordinate of the i'th node, q i' is the azimuth coordinate of the i'th node, p j' is the distance coordinate of the j'th node, q j' is the azimuth coordinate of the j'th node, d i'j' is the Euclidean distance between the i'th node and the j'th node.

[0065] Here, a K-nearest neighbor algorithm may be used to determine multiple neighbor nodes of each node according to the Euclidean distance between nodes, for example, to determine three neighbor nodes of each node.

[0066] In the present invention, the trained deep recognition network includes: a trained sub-aperture feature fusion network, a trained position-aware graph attention network, a trained gated feature fusion module and a trained classifier. Based on this, the above S104 is implemented by the following steps:

[0067] S1041. Using the trained sub-aperture feature fusion network, extract the sub-aperture visual features of each sub-aperture image in a group of sub-aperture images of the measured full-aperture SAR image, and fuse the obtained sub-aperture visual features along the aperture dimension to obtain the visual features of the measured full-aperture SAR image.

[0068] For example, Figure 3 As shown, the sub-aperture feature fusion network includes: a sub-aperture feature extractor and a sub-aperture feature fusion module. The sub-aperture feature extractor is used to extract the sub-aperture visual features of each sub-aperture image in a set of input sub-aperture images. For example, the sub-aperture feature extractor extracts Figure 3 The three sub-aperture images X are input sub1 ,X sub2 ,X sub3 The respective sub-aperture visual features are obtained, thereby obtaining three sub-aperture visual features f sub1 ,f sub2 ,fsub3 The sub-aperture feature fusion module is used to fuse the sub-aperture visual features extracted by the sub-aperture feature extractor along the aperture dimension to obtain a visual feature, for example, Figure 3 As shown in the figure, the sub-aperture feature fusion module combines the three sub-aperture visual features f along the aperture dimension. sub1 ,f sub2 ,f sub3 Fusion is performed to obtain a visual feature f vis . Continue as Figure 3 As shown in , the sub-aperture feature extractor consists of five concatenated convolution blocks, and the sub-aperture feature fusion module is an average pooling layer. Figure 3 As shown, the structures of the first to fourth convolution blocks are the same, including a convolution layer, a batch normalization layer, a maximum pooling layer and an activation function layer, and the fifth convolution block includes a convolution layer, a batch normalization layer and an activation function layer. Exemplarily, the number of convolution kernels of the convolution layers in the first to fifth convolution blocks is 16, 32, 64, 128, and 256, respectively, and the convolution kernel sizes are 5×5, 5×5, 3×3, 3×3, and 4×4, respectively, and the convolution kernel step size is 1; the pooling kernel size of the maximum pooling layer in the first to fourth convolution blocks is 2×2, and the step size is 2; the activation function layers in the first to fifth convolution blocks all use the ReLU activation function. The pooling kernel size of the average pooling layer is 3×1, and the step size is 1.

[0069] The present invention introduces sub-aperture information of SAR images into traditional visual feature extraction, and significantly enhances the identification ability of visual features through sub-aperture decomposition and sub-aperture feature fusion.

[0070] S1042. Use the trained position-aware graph attention network to extract the features of each node in the scattering topology map of the measured full-aperture SAR image, and pool the extracted features to obtain the scattering topology feature vector of the measured full-aperture SAR image.

[0071] Specifically, S1042 is implemented by the following steps:

[0072] S1. Use the trained position-aware graph attention network to perform a first graph convolution processing on the features of each node in the scattering topology map of the measured full-aperture SAR image, and perform a first linear processing on the features of each node in the scattering topology map of the measured full-aperture SAR image, and add the obtained features to obtain a first added feature.

[0073] S2. Activate the first added feature to obtain a first activated feature.

[0074] S3. Perform a second graph convolution process on the first activation feature, perform a second linear process on the first activation feature, and add the obtained features to obtain a second added feature.

[0075] S4. Activate the second added feature to obtain a second activated feature.

[0076] S5. Pool the second activated feature to obtain a scattering topology feature vector of the measured full-aperture SAR image.

[0077] For example, Figure 4 As shown in the figure, the position-aware graph attention network includes: two graph attention convolutional layers, two linear layers, two activation layers and a readout layer. The dimension of the first linear layer is 108×256, and the dimension of the second linear layer is 256×256; both the first graph attention convolutional layer and the second graph attention convolutional layer use 4-head attention, and the features extracted by multiple heads are spliced. In addition, the weight dimension of each convolutional head in the first graph attention convolutional layer is 108×64, and the weight dimension of each convolutional head in the second graph attention convolutional layer is 256×64; both activation layers use the ReLU activation function; the readout layer uses the element-by-element maximum pooling readout method. Continue as Figure 4 As shown in the figure, the input of the first graph attention convolution layer and the first linear layer are the initial features v of each node in the scattering topology graph. i The output of the first graph attention convolution layer and the first linear layer are added as the input of the first activation layer, the output of the first activation layer is used as the input of the second graph attention convolution layer and the second linear layer, the output of the second graph attention convolution layer and the second linear layer are added as the input of the second activation layer, the output of the second activation layer is used as the input of the readout layer, and the output of the readout layer is a scattering topology feature vector f top .

[0078] S1044. Using the trained gated feature fusion module, the visual features and the scattering topology feature vectors are fused through a gating mechanism to obtain the multimodal fusion features of the measured full-aperture SAR image.

[0079] Specifically, S1044 is implemented by the following steps:

[0080] S11. Use the trained gated feature fusion module to perform third linear processing on the visual features to obtain the first visual features.

[0081] S12, performing a Hadamard product on the visual feature and the first visual feature to obtain a transformed visual feature.

[0082] S13, performing a third linear processing on the scattering topology feature vector to obtain a first scattering topology feature.

[0083] S14, performing a Hadamard product on the scattering topology feature vector and the first scattering topology feature to obtain a transformed scattering topology feature vector.

[0084] S15, after splicing the transformed visual features and the transformed scattering topology feature vectors, perform fourth linear processing and activation processing in sequence to obtain multimodal fusion features of the measured full-aperture SAR image.

[0085] The gated multimodal feature fusion module designed in the present invention introduces a gating mechanism to adaptively suppress the conflicting information between modalities and enhance the complementary information between modalities, thereby significantly improving the fusion effectiveness of visual features and scattering topological features.

[0086] For example, Figure 5 As shown in , the gated feature fusion module includes two linear layers and one activation layer. The weight dimensions of the two linear layers are 256×256 and 512×64 respectively, and the activation layer uses the ReLU activation function. Figure 5 As shown, the first input of the first linear layer is a visual feature f vis , the first output is obtained, and the second input of the first linear layer is the scattering topology feature vector f top , and the second output is obtained; the first input and the first output are Hadamard products to obtain the transformed visual features, and the second input and the second output are Hadamard products to obtain the transformed scattering topology feature vector. After that, the transformed visual features and the transformed scattering topology feature vector are concatenated as the input of the second linear layer, and the output of the second linear layer is used as the input of the activation layer. The activation layer outputs the multimodal fusion feature f mul .

[0087] S1045. Using the trained classifier, classify the multimodal fusion features of the measured full-aperture SAR image to obtain the category of the target contained in the measured full-aperture SAR image.

[0088] In some embodiments, the classifier includes a linear layer and a softmax function. Exemplarily, the dimension of the linear layer is 64×10.

[0089] In the present invention, the above-trained deep recognition network is trained using the following method:

[0090] S20: Obtain the above-mentioned preset data set.

[0091] Here, the preset data set is composed of multiple simulated full-aperture SAR images and multiple measured full-aperture SAR images, and the categories of these multiple simulated full-aperture SAR images are consistent with the categories of these multiple measured full-aperture SAR images. For example, the number of categories of these multiple simulated full-aperture SAR images and the number of categories of these multiple measured full-aperture SAR images are both L, and L is a positive integer greater than 1. Each simulated full-aperture SAR image and each measured full-aperture SAR image in the preset data set has a real domain label, and the domain label is used to indicate whether the image is a simulated image or a measured image; in addition, each simulated full-aperture SAR image also has a real category label, indicating the category of the target in the image, and the measured full-aperture SAR image has no real category label.

[0092] S21. Using a sub-aperture decomposition algorithm, each image in the preset data set is divided into a group of sub-aperture images, and a scattering topology map of each image in the preset data set is constructed to obtain a group of sub-aperture images and a scattering topology map for each image in the preset data set.

[0093] The specific sub-aperture decomposition principle and the construction principle of the scattering topology map have been explained in detail in the previous article and will not be repeated here.

[0094] S22. Each time the deep recognition network is trained, I simulated full-aperture SAR images and I measured full-aperture SAR images are selected from the preset data set as 2I training samples; I is a positive integer greater than 0.

[0095] Here, I is a preset value, for example, may be 128. Specifically, I simulated full-aperture SAR images and I measured full-aperture SAR images may be randomly selected from a preset data set.

[0096] S23. Input a group of sub-aperture images and a scattering topology map of each training sample in the 2I training samples into the deep recognition network to be trained this time, and obtain the output of the discriminator and the output of the classifier in the deep recognition network to be trained this time.

[0097] Here, when the deep recognition network is trained for the first time, the deep recognition network to be trained is the initial deep recognition network. The deep recognition network to be trained includes: a sub-aperture feature fusion network to be trained, a position-aware graph attention network to be trained, a gated feature fusion module to be trained, a discriminator to be trained, and a classifier to be trained. The discriminator to be trained is used to predict the domain label of a full-aperture SAR image corresponding to a set of sub-aperture images and a scattering topology map input into the deep recognition network to be trained. The domain label is used to characterize whether the full-aperture SAR image is a simulated image or a measured image. The first layer of the discriminator to be trained is a power-normalized weighted gradient reversal layer. The power-normalized weighted gradient reversal layer is used to perform identity mapping on the input in forward propagation, and invert the input gradient and multiply it by a weight coefficient u in back propagation. i ″, weight coefficient u i The expression of ″ is:

[0098]

[0099] in, represents the domain label of a full-aperture SAR image corresponding to the i-th group of sub-aperture images and the i-th scattering topology map of the deep recognition network to be trained predicted by the discriminator, u i represents the information entropy of the predicted domain label of a full-aperture SAR image corresponding to the i-th group of sub-aperture images and the i-th scattering topology map, α represents the preset power, τ represents the temperature coefficient in the softmax normalization, It represents a set of samples in a batch each time the deep recognition network to be trained is trained. For example, a batch has 128 samples, where one sample is a simulated full-aperture SAR image or a measured full-aperture SAR image. Represents the jth sample in a batch. For example, the value of j is 1, 2, …, 128.

[0100] Specifically, the forward propagation and back propagation formulas of the power normalized weighted gradient reversal layer are:

[0101] PN-WGRL(f mul,i )=f mul,i ;

[0102]

[0103] Where PN-WGRL(·) represents the power normalized weighted gradient reversal layer, f mul,i represents the input of the power-normalized weighted gradient reversal layer in the forward propagation, Indicates the gradient of the input. According to this formula, the power normalized weighted gradient reversal layer PN-WGRL is equivalent to the identity mapping in the forward propagation, while in the backward propagation, the input gradient is inverted and multiplied by a weight coefficient u i ″.

[0104] The present invention designs a power-normalized weighted gradient inversion layer, which can guide the deep recognition network to focus on difficult samples during global alignment, thereby reducing the negative impact of difficult samples on local alignment, significantly improving the cross-domain recognition performance of the deep recognition model, and ultimately improving the recognition accuracy of measured full-aperture SAR images.

[0105] For example, Figure 6 As shown, the discriminator includes: a power normalized weighted gradient reversal layer, three linear processing modules, among which the first linear processing module and the second linear processing module have the same structure and each includes a linear layer, a batch normalization layer and an activation layer in sequence; the third linear processing module includes a linear layer and the first activation layer. The weight dimensions of the linear layers in the three linear processing modules are 64×64, 64×64, and 64×1 respectively; the activation layers in the first linear processing module and the second linear processing module both use the ReLU activation function, and the activation layer in the third linear processing module uses the Sigmoid activation function. Figure 6 As shown, the input of the discriminator is the multimodal fusion feature f mul , the output is the multimodal fusion feature f mul The probability that the corresponding full-aperture SAR image is predicted to be a simulated image Then the multimodal fusion feature f mul The probability that the full-aperture SAR image is predicted to be the measured image The final predicted domain label can be expressed as

[0106] The classifier to be trained includes a linear processing module and a softmax function, where the first linear processing module includes a linear layer with a weight dimension of 64×10. Figure 7 As shown, the input of the classifier is the multimodal fusion feature f mul , the output is the multimodal fusion feature f mul The predicted category probability of the full-aperture SAR image

[0107] S24. Calculate the discrimination loss, classification loss and domain metric loss of this training according to the output of the gated feature fusion module in the deep recognition network to be trained, the output of the discriminator, the output of the classifier, the value I, the real domain labels and real category labels of I simulated training samples during this training, and the real domain labels of I measured training samples.

[0108] In the present invention, the simulated full-aperture SAR image is referred to as source domain data, and thus, the domain where each simulated full-aperture SAR image is located is the source domain, and the measured full-aperture SAR image is referred to as target domain data, and thus, the domain where each measured full-aperture SAR image is located is the target domain. Exemplarily, Figure 7 Schematic diagram of the structure and forward propagation of the deep recognition network of the present invention. Figure 7 As shown, superscript (·) s and(·) t represent the source domain and the target domain respectively, X sub1 ,X sub2 ,X sub3 represents the three sub-aperture images corresponding to any full-aperture SAR image X, is the scattering topology corresponding to any full-aperture SAR image, y and are the true category labels and predicted category labels corresponding to any full-aperture SAR image, d and are the true domain labels and predicted domain labels corresponding to any full-aperture SAR image. Figure 7 The solid line in the figure represents the process of forward propagation of source domain data, and the dotted line represents the process of forward propagation of target domain data. Figure 7 The dotted lines in represent the forward propagation process of source domain data or target domain data in the shared feature extractor. It should be noted that the forward propagation process of source domain data and target domain data is the same, the difference is that only source domain data will participate in the calculation of classification loss. In addition, when calculating the domain metric loss, the source domain uses the true category label and the target domain uses the predicted category label.

[0109] For example, the identification loss uses binary cross entropy loss, and the specific formula is as follows:

[0110]

[0111] Among them, n s and n t represents the number of simulated full-aperture SAR images and measured full-aperture SAR images in each batch, and n s and n t All are I, and represents the true domain label in the form of one-hot encoding of the i-th simulated full-aperture SAR image and the j-th measured full-aperture SAR image in each batch of samples, and Respectively represent the predicted domain labels of the i-th simulated full-aperture SAR image and the j-th measured full-aperture SAR image in each batch of samples; illustratively, in the present invention, d s =[1,0] represents the simulated image, d t =[0,1] represents the measured image.

[0112] For example, the classification loss uses multivariate cross entropy loss, and the specific formula is as follows:

[0113]

[0114] in, represents the category label in the form of one-hot encoding of the i-th simulated full-aperture SAR image in each batch of samples, represents the predicted category label of the i-th simulated full-aperture SAR image.

[0115] Here, by calculating the domain metric loss, the network can be constrained to align the source and target domains class by class in the feature space.

[0116] For example, the domain metric loss uses the local maximum mean discrepancy (LMMD) loss, and the specific formula is as follows:

[0117]

[0118] Where L represents the number of categories of simulated full-aperture SAR images or measured full-aperture SAR images in the preset data set, H is the reproducing kernel Hilbert space defined by the kernel function k(·), represents the multimodal fusion features of the i-th simulated full-aperture SAR image in each batch of samples, represents the multimodal fusion features of the jth simulated full-aperture SAR image in each batch of samples, represents the multimodal fusion features of the i-th measured full-aperture SAR image in each batch of samples, represents the multimodal fusion features of the jth measured full-aperture SAR image in each batch of samples, φ(·) represents the feature mapping of the multimodal fusion features into the reproducing kernel Hilbert space, and Specifically, the kernel function k(·) uses a Gaussian kernel function with the bandwidth set to the average pairwise squared distance between source and target domain samples in the feature space; and They represent the probability that the i-th simulated full-aperture SAR image and the i-th measured full-aperture SAR image belong to the l-th category, respectively. The calculation formula is: in, Represents the i-th simulated full-aperture SAR image in a batch of samples The true one-hot encoded category label vector The lth dimension of Represents the i-th measured full-aperture SAR image in a batch of samples The predicted one-hot encoded class label vector The lth dimension of .

[0119] S25. Calculate the total loss of this training based on the identification loss, classification loss, domain metric loss and weight coefficient of this training.

[0120] Here, the total loss is the sum of the identification loss, classification loss and domain metric loss. The specific formula is as follows:

[0121] L total =Loss dis +Loss cls +λ·Loss met ;

[0122] Where λ is defined as: Among them, epoch current Indicates the current iteration number, epoch total Represents the total number of iterations, and λ will gradually increase as the training progresses.

[0123] S26. Perform back propagation based on the total loss of this training to update the network parameters in the deep recognition network to be trained this time, and obtain the deep recognition network to be trained next time. This iteration is repeated until the number of training times or the degree of convergence of the network meets the preset conditions, and a trained deep recognition network is obtained.

[0124] Here, the preset condition may be a preset number of training times or a preset loss value, which is not limited in the present invention.

[0125] The present invention also provides a UDA SAR ATR device for multimodal feature fusion and global-local domain alignment, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; the memory is used to store a computer program; and the processor is used to implement the steps of the above-mentioned UDA SAR ATR method for multimodal feature fusion and global-local alignment when executing the program stored in the memory.

[0126] Compared with the related methods for identifying measured full-aperture SAR images based on the idea of ​​template matching, the present invention does not require a large number of template images, and since the present invention is a method based on a deep neural network, it is very efficient in identifying images after training. Compared with the related methods that introduce domain adaptation theory and directly input amplitude SAR images into a deep convolutional neural network to extract features for identification, the present invention takes into account the different imaging characteristics of SAR images and optical images, introduces the topological structure knowledge of the target, and thus improves the cross-domain recognition performance. Compared with the related methods that combine SAR domain knowledge and deep learning, the present invention more fully explores the imaging characteristics of SAR images, introduces sub-aperture information in visual feature extraction, introduces the position information of key points in scattering topological feature extraction, and also designs a gated feature fusion module, thereby significantly improving the feature identification ability, and the identification efficiency is high. Compared with the related identification methods based on pseudo-label denoising, the present invention does not require a denoising process when aligning the overall distribution and local distribution between the simulated image and the measured image, and does not require template matching calculation, so the training and recognition efficiency of the present invention are both high. Compared with the method of treating all samples equally, the present invention introduces the idea of ​​difficult sample mining, guiding the model to focus on difficult samples in the global domain alignment, reducing the interference of difficult samples on the subsequent local domain alignment, thereby more accurately aligning the distribution of simulated images and measured images in the feature space, thereby significantly improving the cross-domain recognition performance.

[0127] The present invention adopts a sub-aperture decomposition algorithm to divide a measured full-aperture SAR image into a plurality of different sub-aperture images, and constructs a scattering topology map of the measured full-aperture SAR image at the same time. Afterwards, a trained deep recognition network is used to process the sub-aperture images and the scattering topology map to obtain the category of the target contained in the measured full-aperture SAR image. The present invention introduces the sub-aperture information of SAR images in traditional visual feature extraction, and significantly enhances the ability of visual features to identify targets through sub-aperture feature fusion; the present invention introduces the position information of key points through position encoding when constructing the scattering topology map of the target in the SAR image, and further combines it with the descriptor information of the key points to better model the topological structure of the target, thereby significantly improving the ability of scattering topological features to identify targets; the present invention introduces a gating mechanism when fusing multimodal features, and adaptively mines complementary information between different modal features and suppresses conflicting information through the gating feature fusion module, thereby significantly improving the fusion effect; the present invention introduces the idea of ​​gradient-based difficult sample mining when aligning the global distribution of simulated images and measured images, and further corrects the weight of difficult samples through power transformation and Softmax normalization with temperature, thereby reducing the negative impact of difficult samples on subsequent local domain alignment, and significantly improving the cross-domain recognition performance of the model. In short, the present invention significantly improves the cross-domain recognition performance through multimodal feature fusion and global local domain alignment, thereby improving the recognition accuracy of target categories in measured full-aperture SAR images.

[0128] The effect of the target recognition method of the present invention is further illustrated below through simulation experiments.

[0129] Experimental conditions:

[0130] The hardware platform of the simulation experiment is: the processor is Intel(R) Xeon(R) Silver 4210RCPU@2.40GHz, the memory is 128GB, and the graphics card is GeForce RTX 4090. The software platform is: Ubuntu 20.04 operating system, python3.8, pytorch 1.12 and matlab 2021a.

[0131] Experimental data: The experimental data uses the public SAMPLE dataset, which includes pairs of measured SAR images and simulated SAR images. The dataset is collected by an X-band HH polarimetric radar and includes 10 types of ground vehicle targets, namely: 2S1, BMP2, BTR70, M1, M2, M35, M548, M60, T72 and ZSU23. The resolution of each image is 0.3m×0.3m, the size is 128×128 pixels, the azimuth range is 10°~80°, and the pitch range is 14°~17°. The dataset contains 1345 image pairs, each pair of images contains 1 measured image and 1 simulated image. There are currently two experimental scenarios in this dataset. In the first scenario, the source domain data consists of simulated images of 14°~16°, and the target domain data consists of measured images of 17°. In the second scenario, the source domain data includes all simulated images, and the target domain data includes all measured images. The data details of these two scenarios are shown in Table 1 and Table 2 respectively.

[0132] Table 1 Data details of the first scenario

[0133]

[0134]

[0135] Table 2 Data details of the second scenario

[0136]

[0137]

[0138] Comparative experimental method: The method proposed in the present invention is used to compare the results with multiple existing algorithms under the same conditions. The comparison methods include the literature "A SAR dataset for ATR development: The synthetic and measured paired labeled experiment (SAMPLE)" (Proc. SPIE) published by B. Lewis et al. in 2019, "A deep learning approach to the synthetic and measured paired and labeled experiment (SAMPLE) challenge problem" (Proc. SPIE) published by T. Scarnati et al. in 2019, "Bridging a gap in SAR-ATR: Training on fully synthetic and testing on measured data" (IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens.) published by N. Inkawhich et al. in 2019, "Adversarial training on SAR images" (Proc. SPIE) published by B. Lewis et al. in 2020, and "Ahierarchical receptive network oriented to target recognition in SAR" published by G. Dong et al. in 2022. images”(Pattern Recognit.), Melzer et al., 2021, “Exploring characteristics of neural network architecture computation for enabling SAR ATR” (Proc. SPIE), Sun et al., 2023, “Gradual domain adaptation with pseudo-label denoising for SAR target recognition when using only synthetic data for training” (Remote Sens.), Zhang et al., 2023, "VSFA: Visual and scatteringtopological feature fusion and alignment network for unsupervised domainadaptation in SAR target recognition" (IEEE Trans.Geosci.Remote Sens.).

[0139] Evaluation criteria: SAR image target recognition accuracy.

[0140] Experimental content:

[0141] The method of the present invention and the existing method are used to carry out recognition experiments on the above experimental data, and the comparative experimental results of the present invention and the existing method under two experimental settings are shown.

[0142] Experimental results and analysis:

[0143] Comparative test:

[0144] Table 3 and Table 4 show the recognition accuracy of the method of the present invention and the existing method under experimental setting 1 and experimental setting 2. As shown in Table 3 and Table 4, the method proposed in the present invention is significantly better than the existing method, indicating the effectiveness of the method proposed in the present invention.

[0145] Table 3 Recognition accuracy of each method under experimental setting 1

[0146] method Recognition accuracy Lewis (2019) 24.97％ Scarnati (2019) 55.62％ Inkawhich(2021) 95.06％ Sun(2023) 97.36％ Zhang (2023) 99.15％ The method proposed by the present invention 99.59％

[0147] Table 4 Recognition accuracy of each method under experimental setting 2

[0148] method Recognition accuracy Lewis (2020) 82.05％ Dong(2021) 93.66％ Sun(2023) 96.46％ Melzer(2021) 96.88％ Zhang (2023) 98.18％ The method proposed by the present invention 99.37％

[0149] It should be noted that the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0150] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification.

[0151] In the specification, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude a plurality of components. Certain measures are recorded in different embodiments, but this does not mean that these measures cannot be combined to produce good effects.

[0152] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the protection scope of the present invention.

Claims

1. A UDA SAR ATR method based on multimodal feature fusion and global-local domain alignment, characterized in that: include: Acquire the measured full-aperture SAR image to be identified; Using a sub-aperture decomposition algorithm, the measured full-aperture SAR image is divided into a group of sub-aperture images; The group of sub-aperture images includes a plurality of different sub-aperture images; Constructing a scattering topology map of the measured full-aperture SAR image; Using a trained deep recognition network, the group of sub-aperture images and the scattering topology map of the measured full-aperture SAR image are processed to obtain the category of the target contained in the measured full-aperture SAR image; The trained deep recognition network is obtained by training an initial deep recognition network based on a preset data set; the preset data set contains multiple categories of simulated full-aperture SAR images and multiple categories of measured full-aperture SAR images, and each simulated full-aperture SAR image in the preset data set has a real domain label and a real category label, and each measured full-aperture SAR image has a real domain label, the domain label is used to indicate whether the image is a simulated image or a measured image, and the category label is used to indicate the category of the target in the image.

2. The UDA SAR ATR method of multimodal feature fusion and global-local domain alignment according to claim 1, characterized in that: The trained deep recognition network includes: a trained sub-aperture feature fusion network, a trained position-aware graph attention network, a trained gated feature fusion module and a trained classifier; the trained deep recognition network is used to process the group of sub-aperture images and the scattering topology map of the measured full-aperture SAR image to obtain the category of the target contained in the measured full-aperture SAR image, including: Using a trained sub-aperture feature fusion network, extracting sub-aperture visual features of each sub-aperture image in the group of sub-aperture images of the measured full-aperture SAR image, and fusing the obtained sub-aperture visual features along the aperture dimension to obtain the visual features of the measured full-aperture SAR image; Using a trained position-aware graph attention network, extracting features of each node in the scattering topology graph of the measured full-aperture SAR image, and pooling the extracted features to obtain a scattering topology feature vector of the measured full-aperture SAR image; Using a trained gated feature fusion module, the visual feature and the scattering topology feature vector are fused through a gating mechanism to obtain a multimodal fusion feature of the measured full-aperture SAR image; The trained classifier is used to classify the multimodal fusion features of the measured full-aperture SAR image to obtain the category of the target contained in the measured full-aperture SAR image.

3. The UDA SAR ATR method of multimodal feature fusion and global-local domain alignment according to claim 2, characterized in that: The scattering topology graph includes a plurality of nodes and initial features of each node; The method uses the trained position-aware graph attention network to extract features of each node in the scattering topology graph of the measured full-aperture SAR image, and pools the extracted features to obtain a scattering topology feature vector of the measured full-aperture SAR image, including: Using a trained position-aware graph attention network, performing a first graph convolution process on the initial features of each node in the scattering topology map of the measured full-aperture SAR image, and performing a first linear process on the initial features of each node in the scattering topology map of the measured full-aperture SAR image, and adding the obtained features to obtain a first added feature; Performing activation processing on the first added feature to obtain a first activated feature; Performing a second graph convolution process on the first activation feature, performing a second linear process on the first activation feature, and adding the obtained features to obtain a second added feature; performing activation processing on the second added feature to obtain a second activated feature; The second activated feature is pooled to obtain a scattering topology feature vector of the measured full-aperture SAR image.

4. The UDA SAR ATR method of multimodal feature fusion and global-local domain alignment according to claim 2, characterized in that: The trained gated feature fusion module is used to fuse the visual features and the scattering topology feature vector through a gated mechanism to obtain the multimodal fusion features of the measured full-aperture SAR image, including: Using a trained gated feature fusion module, performing a third linear process on the visual feature to obtain a first visual feature; Performing a Hadamard product on the visual feature and the first visual feature to obtain a transformed visual feature; Performing a third linear processing on the scattering topology feature vector to obtain a first scattering topology feature; Performing a Hadamard product on the scattering topology feature vector and the first scattering topology feature to obtain a transformed scattering topology feature vector; After the transformed visual features and the transformed scattering topology feature vectors are spliced, fourth linear processing and activation processing are performed in sequence to obtain the multimodal fusion features of the measured full-aperture SAR image.

5. The UDA SAR ATR method of multimodal feature fusion and global-local domain alignment according to claim 1, characterized in that: The initial deep recognition network includes: an initial sub-aperture feature fusion network, an initial position-aware graph attention network, an initial gated feature fusion module, an initial discriminator and an initial classifier; wherein the initial discriminator is used to predict the domain labels of a full-aperture SAR image corresponding to a set of sub-aperture images and a scattering topology map input into the initial deep recognition network, and the domain labels are used to characterize whether the full-aperture SAR image is a simulated image or a measured image; the first layer of the initial discriminator is a power-normalized weighted gradient reversal layer, and the power-normalized weighted gradient reversal layer is used to perform identity mapping on the input in forward propagation, and invert the input gradient and multiply it by a weight coefficient u in back propagation. i ″, the weight coefficient u i The expression of ″ is: in, represents the domain label of a full-aperture SAR image corresponding to the i-th group of sub-aperture images and the i-th scattering topology map input to the initial deep recognition network predicted by the discriminator, u i represents the information entropy of the predicted domain label of a full-aperture SAR image corresponding to the i-th group of sub-aperture images and the i-th scattering topology map, α represents the preset power, τ represents the temperature coefficient in the softmax normalization, It represents a set of samples in a batch each time the network is trained, where a sample is a simulated full-aperture SAR image or a measured full-aperture SAR image.

6. The UDA SAR ATR method of multimodal fusion feature fusion and global local domain alignment according to claim 1, characterized in that: The sub-aperture decomposition algorithm is used to divide the measured full-aperture SAR image into a group of sub-aperture images, including: Performing a one-dimensional Fourier transform on the measured full-aperture SAR image along the azimuth direction to convert the measured full-aperture SAR image into a range-Doppler domain to obtain a Doppler domain image; Performing a zero removal operation and a window removal operation on the Doppler domain image in sequence to obtain a window removal image; Divide the spectrum of the windowed image into three parts at equal intervals along the azimuth direction to obtain three sub-spectra, wherein the three sub-spectra have overlapping parts, and each sub-spectra contains 1 / 2 of the original spectrum of the windowed image; Each sub-spectrum is sequentially subjected to windowing, zero-filling, one-dimensional inverse Fourier transform, quarter-power amplitude normalization and center cropping operations to obtain a sub-aperture image of a preset size; the three sub-aperture images corresponding to the three sub-spectra constitute a group of sub-aperture images of the measured full-aperture SAR image.

7. The UDA SAR ATR method of multimodal fusion feature fusion and global local domain alignment according to claim 1, characterized in that: The step of constructing the scattering topology map of the measured full-aperture SAR image comprises: Performing contrast enhancement processing on the measured full-aperture SAR image to obtain a processed image; Extracting scattering topological points of the processed image, and generating a descriptor for each extracted scattering topological point; each extracted scattering topological point has a two-dimensional space coordinate, and the two-dimensional space coordinate includes: a range coordinate and an azimuth coordinate; For each scattering topological point, generating an initial feature of the scattering topological point according to a two-dimensional spatial coordinate and a descriptor of the scattering topological point; The scattering topological point is taken as a node, the initial feature of the scattering topological point is taken as the initial feature of the node, and multiple neighbor nodes of each node are determined according to the Euclidean distance between nodes, and a unidirectional edge is generated from each node to each neighbor node of the node, so as to obtain the scattering topological map of the measured full-aperture SAR image.

8. The UDA SAR ATR method of multimodal feature fusion and global-local domain alignment according to claim 7, characterized in that: The two-dimensional space coordinates include range coordinates and azimuth coordinates; generating the initial features of the scattering topological point according to the two-dimensional space coordinates and a descriptor of the scattering topological point includes: Respectively calculating the K-bit position codes of the distance coordinates and the K-bit position codes of the azimuth coordinates of the scattering topological point, where K is a preset integer; splicing the K-bit position codes of the range coordinates and the K-bit position codes of the azimuth coordinates to obtain the position codes of the two-dimensional space coordinates of the scattering topological point; The average value of the descriptor of the scattering topological point and the position encoding of the two-dimensional space coordinates of the scattering topological point is calculated, and the average value is used as the initial feature of the scattering topological point.

9. The UDA SAR ATR method of multimodal fusion feature fusion and global local domain alignment according to claim 1, characterized in that: The trained deep recognition network is trained using the following method: Acquire the preset data set; Using a sub-aperture decomposition algorithm, each image in the preset data set is divided into a group of sub-aperture images, and a scattering topology map of each image in the preset data set is constructed to obtain a group of sub-aperture images and a scattering topology map for each image in the preset data set; Each time the deep recognition network is trained, I simulated full-aperture SAR images and I measured full-aperture SAR images are selected from the preset data set as 2I training samples; I is a positive integer greater than 0; A set of sub-aperture images and a scattering topology map of each training sample in the 2I training samples are input into the deep recognition network to be trained this time, and an output of the discriminator and an output of the classifier in the deep recognition network to be trained this time are obtained; According to the output of the gated feature fusion module in the deep recognition network to be trained, the output of the discriminator, the output of the classifier, the value I, the real domain labels and real category labels of I simulated training samples during this training, and the real domain labels of I measured training samples, calculate the discrimination loss, classification loss and domain metric loss of this training; Calculate the total loss of this training based on the identification loss, classification loss, domain metric loss and weight coefficient of this training; Back propagation is performed based on the total loss of this training to update the network parameters in the deep recognition network to be trained this time, and the deep recognition network to be trained next time is obtained. This iteration is repeated until the number of training times or the degree of convergence of the network meets the preset conditions, and the trained deep recognition network is obtained.

10. A UDA SAR ATR device with multi-modal fusion feature fusion and global local domain alignment, comprising a processor, a communication interface, a memory and a communication bus, characterized in that: The processor, the communication interface and the memory communicate with each other via the communication bus; The memory is used to store computer programs; The processor is used to implement the method steps described in any one of claims 1-9 when executing the program stored in the memory.

Images