UDA sar atr method and device based on multi-modal feature fusion and global-local domain alignment

Abstract

The application discloses a kind of multimodal feature fusion and global local domain alignment UDA SAR ATR method and equipment, comprising: obtaining the full-aperture SAR image to be identified of actual measurement;The image is divided into a group of sub-aperture images;The scattering topological graph of the image is constructed;Using the trained deep recognition network, a group of sub-aperture images and scattering topological graph of the image are processed, to obtain the category of target contained in the image;The trained deep recognition network is obtained based on preset data set training;Multiple categories of simulation full-aperture SAR images and actual measurement full-aperture SAR images are contained in preset data set, each simulation full-aperture SAR image has real domain label and category label, each actual measurement full-aperture SAR image only has real domain label, domain label indicates whether image is simulation image or actual measurement image, and category label indicates the category of target in image.The application can improve cross-domain SAR target recognition performance.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of radar image processing, and particularly relates to a multi-modal feature fusion and global-local domain alignment UDA SAR ATR method and device. BACKGROUND

[0002] Synthetic Aperture Radar (SAR) adopts the mode of active microwave emission and has all-weather and all-time imaging capability. Therefore, SAR has become an important device for earth observation and has a wide range of applications in military defense, environmental monitoring, disaster warning and urban planning. In recent years, automatic target recognition (ATR) has gradually become one of the most important research topics in the field of SAR. In the past decade, deep learning has achieved great success in the field of computer vision. Therefore, more and more scholars have introduced deep learning into SAR ATR and have proposed a variety of excellent recognition models. As a typical data-driven method, a model based on deep learning usually needs a large amount of data to effectively capture the complex features and patterns in the data. However, compared with optical images, the acquisition and labeling of SAR images are more difficult, which brings challenges to the application of deep learning in SAR ATR. Given the special imaging mechanism of SAR, computer-aided design (CAD) modeling and electromagnetic calculation to generate simulated SAR images is a feasible method to alleviate the lack of data. However, due to the approximation and simplification in the process of electromagnetic simulation and model design, there is a non-negligible distribution difference between simulated and measured SAR images, which makes the recognition model trained on simulated images not well generalized to measured images. Therefore, how to cross the distribution difference between simulated and measured images and accurately recognize the measured images based on simulated images has become an engineering problem with practical application value. In view of this problem, there are currently some solutions based on the theory of unsupervised domain adaptation (UDA), but the cross-domain recognition performance of these methods is still not good enough, so that the recognition result of the class of the target in the measured full-aperture SAR image is not accurate enough. SUMMARY

[0003] In order to solve the above problems existing in the prior art, the application provides a multi-modal feature fusion and global-local domain alignment UDA SAR ATR method and device.

[0004] The technical problem to be solved by the application is solved by the following technical scheme:

[0005] The application provides a multi-modal feature fusion and global-local domain alignment UDA SAR ATR method, comprising:

[0006] acquiring a measured full-aperture SAR image to be recognized;

[0007] The measured full-aperture SAR image is divided into a group of sub-aperture images by using a sub-aperture decomposition algorithm; the group of sub-aperture images includes multiple different sub-aperture images;

[0008] A scattering topology graph of the measured full-aperture SAR image is constructed;

[0009] The group of sub-aperture images and the scattering topology graph of the measured full-aperture SAR image are processed by using the trained deep recognition network to obtain the category of the target contained in the measured full-aperture SAR image;

[0010] The trained deep recognition network is obtained by training an initial deep recognition network based on a preset data set; the preset data set contains simulation full-aperture SAR images of multiple categories and measured full-aperture SAR images of multiple categories, and each simulation full-aperture SAR image in the preset data set has a real domain label and a real category label, and each measured full-aperture SAR image has a real domain label, the domain label is used to indicate whether the image is a simulation image or a measured image, and the category label is used to indicate the category of the target in the image.

[0011] The application further provides a multi-modal feature fusion and global-local domain alignment UDA SAR ATR device, which comprises a processor, a communication interface, a memory and a communication bus, the processor, the communication interface and the memory complete communication with each other through the communication bus; the memory is used for storing a computer program; the processor is used for executing the program stored on the memory, and the steps of the multi-modal feature fusion and global-local domain alignment UDA SAR ATR method are realized.

[0012] Compared with the prior art, the application has the following beneficial effects:

[0013] The application adopts a sub-aperture decomposition algorithm to divide a measured full-aperture SAR image into multiple different sub-aperture images, and simultaneously constructs a scattering topological graph of the measured full-aperture SAR image, then adopts a trained deep recognition network to process the sub-aperture images and the scattering topological graph to obtain the category of a target contained in the measured full-aperture SAR image. The application introduces sub-aperture information of a SAR image into traditional visual feature extraction, and significantly enhances the discrimination ability of visual features to targets through sub-aperture feature fusion; the application introduces position information of key points through position coding when constructing a scattering topological graph of a target in a SAR image, and further combines with the descriptor information of the key points to better model the topological structure of the target, thereby significantly enhancing the discrimination ability of scattering topological features to targets; the application introduces a gating mechanism when fusing multi-modal features, and adaptively excavates complementary information and suppresses conflicting information between different modal features through a gating feature fusion module, thereby significantly improving the fusion effect; the application introduces a gradient-based difficult sample mining idea when aligning the global distribution of the simulation image and the measured image, and further corrects the weight of the difficult sample through power transformation and temperature-based Softmax normalization, thereby reducing the negative impact of the difficult sample on subsequent local domain alignment, and significantly improving the cross-domain recognition performance of the model. In summary, the application significantly improves the cross-domain recognition performance through multi-modal feature fusion and global-local domain alignment, thereby improving the recognition accuracy of the target category in the measured full-aperture SAR image.

[0014] The application will be further described in detail below in combination with the drawings and specific embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1 is a flowchart of a multi-modal feature fusion and global-local domain alignment UDA SARATR method provided by an embodiment of the application;

[0016] Figure 2 is a schematic diagram of a sub-aperture decomposition process provided by an embodiment of the application;

[0017] Figure 3 is a structural schematic diagram of a sub-aperture feature fusion network provided by an embodiment of the application;

[0018] Figure 4 is a structural schematic diagram of a position-aware graph attention network provided by an embodiment of the application;

[0019] Figure 5 is a structural schematic diagram of a gating feature fusion module provided by an embodiment of the application;

[0020] Figure 6 is a structural schematic diagram of a discriminator provided by an embodiment of the application;

[0021] Figure 7 is a schematic diagram of the structure and forward propagation of the deep recognition network provided by an embodiment of the present application. DETAILED DESCRIPTION

[0022] The present application will be further described in detail below in conjunction with specific embodiments, but the embodiments of the present application are not limited thereto.

[0023] Figure 1 is a flowchart of a multi-modal feature fusion and global-local domain alignment UDA SARATR method provided by an embodiment of the present application, as shown in the figure, the method comprises: Figure 1

[0024] S101, acquiring a measured full-aperture SAR image to be recognized.

[0025] Here, the measured full-aperture SAR image to be recognized can be one or multiple, and the present application does not limit this, and the processing steps in S102-S104 below are steps for processing each measured full-aperture SAR image.

[0026] S102, using a sub-aperture decomposition algorithm to divide the measured full-aperture SAR image into a group of sub-aperture images; the group of sub-aperture images includes multiple different sub-aperture images.

[0027] S103, constructing a scattering topological graph of the measured full-aperture SAR image.

[0028] S104, using a trained deep recognition network to process the group of sub-aperture images and the scattering topological graph of the measured full-aperture SAR image to obtain the category of the target contained in the measured full-aperture SAR image; wherein the trained deep recognition network is obtained by training an initial deep recognition network based on a preset data set; the preset data set contains multiple categories of simulated full-aperture SAR images and multiple categories of measured full-aperture SAR images, and each simulated full-aperture SAR image in the preset data set has a real domain label and a real category label, and each measured full-aperture SAR image only has a real domain label, the domain label is used to indicate whether the image is a simulated image or a measured image, and the category label is used to indicate the category of the target in the image.

[0029] In some embodiments, S102 above is implemented by the following steps:

[0030] S1021, performing one-dimensional Fourier transform on the measured full-aperture SAR image along the azimuth direction to convert the measured full-aperture SAR image to the range-Doppler domain to obtain a Doppler domain image.

[0031] S1022, sequentially performing zero removal operation and window removal operation on the Doppler domain image to obtain a window-removed image.​

[0032] S1023. Divide the spectrum of the windowed image into three equal parts along the azimuth direction to obtain three sub-spectrums. There are overlapping parts in the three sub-spectrums, and each sub-spectrum contains 1 / 2 of the original spectrum of the windowed image.

[0033] S1024. For each sub-spectrum, perform windowing, zero-filling, one-dimensional inverse Fourier transform, quarter power magnitude (QPM) normalization, and center clipping operations in sequence to obtain a sub-aperture image of a preset size; the three sub-spectrums correspond one-to-one to the three sub-aperture images, which constitute a set of sub-aperture images of the measured full-aperture SAR image.

[0034] For example, such as Figure 2 As shown, a full-aperture SAR image X is divided into a group of sub-aperture images X. sub1 ,X sub2 ,X sub3 The steps are as follows:

[0035] (1a) Perform a one-dimensional Fourier transform on the full aperture SAR image along the azimuth direction to convert the image to the range-Doppler domain and obtain a Doppler domain image.

[0036] (1b) Remove the zero elements added to the spectrum in order to smooth the image in the Doppler domain image to obtain the zero-removed image;

[0037] (1c) Multiply the spectrum of the zero-removed image by the inverse of the preset window function to remove the influence of the window function and obtain the de-windowed image; for example, the preset window function is a Taylor window function with a peak-to-sidelobe ratio of -35dB.

[0038] (1d) Divide the spectrum of the windowed image into three equal parts along the azimuth direction to obtain three sub-spectrums. There are overlapping parts in the three sub-spectrums, and each sub-spectrum contains 1 / 2 of the original spectrum of the windowed image.

[0039] (1e) Windowing is performed independently for each sub-spectrum. For example, a Taylor window with a peak-to-sidelobe ratio of -35dB is used to suppress sidelobes and obtain windowed sub-spectrums.

[0040] (1f) Perform zero-padding on the windowed spectrum to increase the number of sampling points to smooth the image and obtain the zero-padding sub-spectrum;

[0041] (1g) Perform a one-dimensional inverse Fourier transform on the zero-padded sub-spectrum to transform the spectrum to the time domain and obtain a sub-aperture image, which is a complex-valued sub-aperture image.

[0042] (1h) independently performing quarter-power amplitude normalization on each sub-aperture image to enhance the contrast thereof;

[0043] Specifically, the specific steps of the quarter-power amplitude normalization are as follows:

[0044] (1h1) taking modulus of the sub-aperture image to obtain an amplitude sub-aperture image;

[0045] (1h2) assigning the amplitude value of each pixel point of the amplitude sub-aperture image to 1 if the amplitude value of the pixel point is greater than 1 to obtain a truncated image;

[0046] (1h3) performing a preset power transformation on the truncated image to enhance the contrast thereof; for example, the preset power is 0.5;

[0047] (1h4) using maximum-minimum normalization to normalize the pixel amplitude value range of the amplitude sub-aperture image after the power transformation to [0, 1];

[0048] (1i) independently performing center cropping on each sub-aperture image after the contrast enhancement to crop the sub-aperture image to a preset size to obtain a final set of sub-aperture images X sub1 ,X sub2 ,X sub3 For example, the preset size is 64x64 pixels.

[0049] In some embodiments, the "constructing a scattering topology map of the measured full-aperture SAR image" in S103 is implemented by the following steps:

[0050] S1031, performing contrast enhancement processing on the measured full-aperture SAR image to obtain a processed image.

[0051] For example, quarter-power amplitude normalization can be performed on the measured full-aperture SAR image to enhance the contrast of the measured full-aperture SAR image to obtain the processed image.

[0052] S1032, extracting scattering topology points of the processed image, and generating a description sub for each extracted scattering topology point. Each extracted scattering topology point has a two-dimensional spatial coordinate, which includes a range coordinate and an azimuth coordinate.

[0053] For example, the SAR-SIFT algorithm can be used to extract the scattering topology points of the processed image, and each scattering topology point includes three attributes: a description sub r describing the local backscattering characteristics in the form of a gradient, a range coordinate p, and an azimuth coordinate q.

[0054] S1033、for each scattering topological point, generating an initial feature of the scattering topological point according to one two-dimensional spatial coordinate of the scattering topological point and one descriptor of the scattering topological point.

[0055] The above S1033 is realized by the following steps:

[0056] A1、for each scattering topological point in a scattering topological map of a measured full-aperture SAR image, respectively calculating K-bit position encoding of a range coordinate of the scattering topological point and K-bit position encoding of an azimuth coordinate of the scattering topological point, K being a preset integer.

[0057] Here, the kth bit encoding (kth element) in the K-bit position encoding of the range coordinate of a scattering topological point or the kth bit encoding in the K-bit position encoding of the azimuth coordinate of the scattering topological point is calculated according to the following formula: wherein k is 1, 2, …, K, even represents even number, odd represents odd number, and pos represents the range coordinate or the azimuth coordinate of the scattering topological point.

[0058] A2、splicing the K-bit position encoding of the range coordinate and the K-bit position encoding of the azimuth coordinate to obtain position encoding of the two-dimensional spatial coordinate of the scattering topological point.

[0059] Exemplarily, the expression of the position encoding e of the two-dimensional spatial coordinate of a scattering topological point is e=PE(p)||PE(q), PE(p) representing the K-bit position encoding of the range coordinate of the scattering topological point, PE(q) representing the K-bit position encoding of the azimuth coordinate of the scattering topological point, and || representing splicing.

[0060] A3、calculating an average of the descriptor of the scattering topological point and the position encoding of the two-dimensional spatial coordinate of the scattering topological point, and taking the average as the initial feature of the scattering topological point.

[0061] Exemplarily, the expression of the initial feature v of a scattering topological point is v=(e+r) / 2.

[0062] In the scattering topological feature extraction based on the SAR-SIFT algorithm, the position information and the descriptor information of the key point are simultaneously used, and the two-dimensional position information is mapped to a high-dimensional space through the position encoding theory, so that the scattering topological feature can be effectively combined with the descriptor information, and the discrimination ability of the scattering topological feature to the target is significantly improved.

[0063] S1034. Treat each scattering topological point as a node, use the initial features of the scattering topological point as the initial features of the node, and determine multiple neighboring nodes of each node based on the Euclidean distance between nodes. Generate a one-way edge from each node to each of its neighboring nodes, thus obtaining a set of nodes of the measured full-aperture SAR image. The scattering topology graph formed by the set of edges ε

[0064] Here, the Euclidean distance between any two nodes can be calculated using two-dimensional spatial coordinates. For example, the formula for calculating the Euclidean distance between any two nodes is: Where the i'-th node and the j'-th node are any two nodes, p i' Let q be the distance coordinate of the i'-th node. i' Let p be the orientation coordinate of the i'th node. j' Let q be the distance coordinate of the j'-th node. j' Let d be the orientation coordinate of the j'-th node. i'j' Let be the Euclidean distance between the i'th node and the j'th node.

[0065] Here, the K-nearest neighbor algorithm can be used to determine multiple neighboring nodes of each node based on the Euclidean distance between nodes. For example, it can determine 3 neighboring nodes of each node.

[0066] In this invention, the trained deep recognition network includes: a trained sub-aperture feature fusion network, a trained position-aware graph attention network, a trained gated feature fusion module, and a trained classifier. Based on this, S104 is implemented through the following steps:

[0067] S1041. Using a trained sub-aperture feature fusion network, extract the sub-aperture visual features of each sub-aperture image from a set of sub-aperture images of the measured full-aperture SAR image, and fuse the obtained sub-aperture visual features along the aperture dimension to obtain the visual features of the measured full-aperture SAR image.

[0068] For example, such as Figure 3 As shown, the sub-aperture feature fusion network includes a sub-aperture feature extractor and a sub-aperture feature fusion module. The sub-aperture feature extractor is used to extract the sub-aperture visual features of each sub-aperture image in a set of input sub-aperture images. For example, the sub-aperture feature extractor extracts... Figure 3 The three sub-aperture images X input to the image sub1 ,X sub2 ,X sub3 Each sub-aperture visual feature is used to obtain three sub-aperture visual features f. sub1 ,f sub2 ,fsub3 The sub-aperture feature fusion module is used to fuse the sub-aperture visual features extracted by the sub-aperture feature extractor along the aperture dimension to obtain a single visual feature, such as... Figure 3 As shown, the sub-aperture feature fusion module integrates the visual features f of the three sub-apertures along the aperture dimension. sub1 ,f sub2 ,f sub3 By fusing the components, a visual feature f is obtained. vis Continue as follows Figure 3 As shown, the sub-aperture feature extractor consists of five concatenated convolutional blocks, and the sub-aperture feature fusion module is an average pooling layer. Continuing as... Figure 3 As shown, the structures of the first to fourth convolutional blocks are identical, each including a convolutional layer, a batch normalization layer, a max pooling layer, and an activation function layer. The fifth convolutional block includes a convolutional layer, a batch normalization layer, and an activation function layer. For example, the number of convolutional kernels in the convolutional layers of the first to fifth convolutional blocks are 16, 32, 64, 128, and 256 respectively, with kernel sizes of 5×5, 5×5, 3×3, 3×3, and 4×4 respectively, and a stride of 1 for all kernels. The max pooling layers in the first to fourth convolutional blocks all have a 2×2 kernel size and a stride of 2. The activation function layers in the first to fifth convolutional blocks all use the ReLU activation function. The average pooling layer has a 3×1 kernel size and a stride of 1.

[0069] This invention introduces sub-aperture information from SAR images into traditional visual feature extraction, and significantly enhances the ability to identify visual features through sub-aperture decomposition and sub-aperture feature fusion.

[0070] S1042. Using a trained position-aware graph attention network, extract the features of each node in the scattering topology graph of the measured full-aperture SAR image, and pool the extracted features to obtain the scattering topology feature vector of the measured full-aperture SAR image.

[0071] Specifically, S1042 is implemented through the following steps:

[0072] S1. Using a trained position-aware graph attention network, perform a first graph convolution process on the features of each node in the scattering topology map of the measured full-aperture SAR image, and perform a first linear process on the features of each node in the scattering topology map of the measured full-aperture SAR image. Then, add the obtained features to obtain the first summed features.

[0073] S2. Activate the first additive feature to obtain the first activated feature.

[0074] S3, performing second graph convolution processing on the first activation feature, and performing second linear processing on the first activation feature, and adding the obtained features to obtain a second added feature.

[0075] S4, performing activation processing on the second added feature to obtain a second activation feature.

[0076] S5, performing pooling on the second activation feature to obtain a scattering topological feature vector of the measured full-aperture SAR image.

[0077] As shown in Figure 4 , the position-aware graph attention network includes two graph attention convolution layers, two linear layers, two activation layers, and a readout layer. The dimension of the first linear layer is 108x256, and the dimension of the second linear layer is 256x256; the first graph attention convolution layer and the second graph attention convolution layer both use 4 heads of attention, and the features extracted by multiple heads are spliced, and the weight dimension of each convolution head in the first graph attention convolution layer is 108x64, and the weight dimension of each convolution head in the second graph attention convolution layer is 256x64; both activation layers use ReLU activation function; the readout layer uses element-wise maximum pooling readout method. As shown in Figure 4 , the input of the first graph attention convolution layer and the first linear layer is the initial feature v i of each node in the scattering topological graph, and the output of the first graph attention convolution layer and the first linear layer is added to serve as the input of the first activation layer, the output of the first activation layer serves as the input of the second graph attention convolution layer and the second linear layer, the output of the second graph attention convolution layer and the second linear layer is added to serve as the input of the second activation layer, the output of the second activation layer serves as the input of the readout layer, and the output of the readout layer is a scattering topological feature vector f top .

[0078] S1044, using the trained gating feature fusion module to fuse the visual feature and the scattering topological feature vector through a gating mechanism to obtain a multi-modal fusion feature of the measured full-aperture SAR image.

[0079] Specifically, S1044 is implemented by the following steps:

[0080] S11, using the trained gating feature fusion module to perform third linear processing on the visual feature to obtain a first visual feature.

[0081] S12, performing Hadamard product on the visual feature and the first visual feature to obtain a transformed visual feature.

[0082] S13, performing third linear processing on the scattering topological feature vector to obtain a first scattering topological feature.

[0083] S14, perform Hadamard product on the scattering topological feature vector and the first scattering topological feature to obtain a transformed scattering topological feature vector.

[0084] S15, perform splicing on the transformed visual feature and the transformed scattering topological feature vector, and then sequentially perform fourth linear processing and activation processing to obtain a multi-modal fusion feature of the measured full-aperture SAR image.

[0085] The gate multi-modal feature fusion module designed in the application can adaptively suppress the information of inter-modal conflicts and enhance the information of inter-modal complementation by introducing a gate mechanism, thereby significantly improving the fusion effectiveness of the visual feature and the scattering topological feature.

[0086] As shown in FIG. 1, the gate feature fusion module includes two linear layers and an activation layer. Figure 5 As shown in FIG. 1, the gate feature fusion module includes two linear layers and an activation layer. Figure 5 As shown in FIG. 1, the first input of the first linear layer is a visual feature f vis , and the first output is obtained, and the second input of the first linear layer is a scattering topological feature vector f top , and the second output is obtained; the first input and the first output are subjected to Hadamard product to obtain a transformed visual feature, and the second input and the second output are subjected to Hadamard product to obtain a transformed scattering topological feature vector, and then the transformed visual feature and the transformed scattering topological feature vector are spliced to serve as the input of the second linear layer, the output of the second linear layer serves as the input of the activation layer, and the activation layer outputs a multi-modal fusion feature f mul .

[0087] S1045, using the trained classifier, performing classification processing on the multi-modal fusion feature of the measured full-aperture SAR image to obtain the category of the target contained in the measured full-aperture SAR image.

[0088] In some embodiments, the classifier includes a linear layer and a softmax function, and the dimension of the linear layer is 64x10.

[0089] In the application, the above-described trained deep recognition network is obtained by using the following method for training:

[0090] S20, obtaining the above-described preset data set.

[0091] Here, the preset data set is composed of multiple simulation full-aperture SAR images and multiple measured full-aperture SAR images, and the categories of the multiple simulation full-aperture SAR images are consistent with the categories of the multiple measured full-aperture SAR images, for example, the number of the categories of the multiple simulation full-aperture SAR images is L, and the number of the categories of the multiple measured full-aperture SAR images is also L, where L is a positive integer greater than 1. Each simulation full-aperture SAR image and each measured full-aperture SAR image in the preset data set has a real domain label, which is used to indicate whether the image is a simulation image or a measured image; in addition, each simulation full-aperture SAR image has a real category label, which indicates the category of the target in the image, and the measured full-aperture SAR image does not have a real category label.

[0092] S21, each image in the preset data set is divided into a group of sub-aperture images by using a sub-aperture decomposition algorithm, and a scattering topology graph of each image in the preset data set is constructed, so as to obtain a group of sub-aperture images and a scattering topology graph of each image in the preset data set.

[0093] The specific sub-aperture decomposition principle and the construction principle of the scattering topology graph have been described in detail in the foregoing, and will not be repeated here.

[0094] S22, when training the deep recognition network each time, I simulation full-aperture SAR images and I measured full-aperture SAR images are selected from the preset data set as 2I training samples; I is a positive integer greater than 0.

[0095] Here, I is a preset value, for example, can be 128. Specifically, I simulation full-aperture SAR images and I measured full-aperture SAR images can be randomly selected from the preset data set.

[0096] S23, a group of sub-aperture images and a scattering topology graph of each training sample in the 2I training samples are input into the deep recognition network to be trained this time, so as to obtain the output of the discriminator and the output of the classifier in the deep recognition network to be trained this time.

[0097] Here, when the deep recognition network is first trained, the deep recognition network to be trained is an initial deep recognition network. The deep recognition network to be trained includes: a sub-aperture feature fusion network to be trained, a position-aware graph attention network to be trained, a gated feature fusion module to be trained, a discriminator to be trained, and a classifier to be trained. The discriminator to be trained is used to predict a domain label of a full-aperture SAR image corresponding to a group of sub-aperture images and a scatter topological map input into the deep recognition network to be trained, and the domain label is used to represent whether the full-aperture SAR image is a simulation image or a measured image. The first layer of the discriminator to be trained is a power normalization weighted gradient reversal layer, which is used to perform identity mapping on input in forward propagation, and to take the negative of the gradient of the input and multiply it by a weight coefficient u i The expression of the weight coefficient u i is as follows:

[0098]

[0099] wherein, represents the domain label of the i-th full-aperture SAR image corresponding to the i-th group of sub-aperture images and the i-th scatter topological map predicted by the discriminator, u i represents the information entropy of the predicted domain label of the i-th full-aperture SAR image corresponding to the i-th group of sub-aperture images and the i-th scatter topological map, a represents a preset power, and τ represents a temperature coefficient in softmax normalization, represents a set of samples in a batch each time the deep recognition network to be trained is trained, for example, there are 128 samples in a batch, wherein a sample is a simulation full-aperture SAR image or a measured full-aperture SAR image, represents the j-th sample in a batch, for example, j takes values of 1, 2, …, 128.

[0100] Specifically, the forward propagation and the backward propagation formulas of the power normalization weighted gradient reversal layer are as follows:

[0101] PN-WGRL(f mul,i )=f mul,i ;

[0102]

[0103] wherein, PN-WGRL(·) represents the power normalization weighted gradient reversal layer, f mul,i represents the input of the power normalization weighted gradient reversal layer in forward propagation, and This indicates that the gradient of the input is calculated. According to this formula, the Power Normalized Weighted Gradient Reversal Layer (PN-WGRL) is equivalent to an identity mapping in forward propagation, while in backward propagation it inverts the gradient of the input and multiplies it by a weight coefficient u. i ″.

[0104] This invention, through the design of a power-normalized weighted gradient inversion layer, guides the deep recognition network to focus on difficult samples during global alignment, thereby reducing the negative impact of difficult samples on local alignment, significantly improving the cross-domain recognition performance of the deep recognition model, and ultimately improving the recognition accuracy of measured full-aperture SAR images.

[0105] For example, such as Figure 6 As shown, the discriminator sequentially includes: a power-normalized weighted gradient inversion layer and three linear processing modules. The first and second linear processing modules have identical structures, each consisting of a linear layer, a batch normalization layer, and an activation layer. The third linear processing module includes a linear layer and a first activation layer. The weight dimensions of the linear layers in the three linear processing modules are 64×64, 64×64, and 64×1, respectively. The activation layers in the first and second linear processing modules use the ReLU activation function, while the activation layer in the third linear processing module uses the Sigmoid activation function. Figure 6 As shown, the input to the discriminator is the multimodal fusion feature f. mul The output is a multimodal fusion feature f. mul The probability that the full-aperture SAR image is predicted as a simulated image. Then the multimodal fusion feature f mul The probability that the full-aperture SAR image is predicted to be the actual measured image. The final predicted domain label can be represented as

[0106] The classifier to be trained consists of a linear processing module and a softmax function. The first linear processing module includes a linear layer with a weight dimension of 64×10. Figure 7 As shown, the input to the classifier is the multimodal fusion feature f. mul The output is a multimodal fusion feature f. mul Predicted category probability of the full-aperture SAR image

[0107] S24, according to the output of the gating feature fusion module in the deep recognition network to be trained this time, the output of the discriminator, the output of the classifier, the numerical value I, the real domain label and the real category label of the I simulation training samples in this training, and the real domain label of the I measured training samples, calculate the discrimination loss, the classification loss and the field measurement loss of this training.

[0108] In the present application, the simulation full-aperture SAR image is referred to as source domain data, so the domain of each simulation full-aperture SAR image is the source domain, and the measured full-aperture SAR image is referred to as target domain data, so the domain of each measured full-aperture SAR image is the target domain. Exemplarily, Figure 7 is a schematic diagram of the structure and forward propagation of the deep recognition network of the present application. As shown in Figure 7 , the superscripts (·) s and (·) t respectively represent the source domain and the target domain, X sub1 , X sub2 , X sub3 represent three sub-aperture images corresponding to any one full-aperture SAR image X, is the scattering topology graph corresponding to the arbitrary full-aperture SAR image, y and is the real category label and the predicted category label corresponding to the arbitrary full-aperture SAR image, d and is the real domain label and the predicted domain label corresponding to the arbitrary full-aperture SAR image. Figure 7 The solid line in indicates the forward propagation process of the source domain data, and the dashed line indicates the forward propagation process of the target domain data, Figure 7 The dotted line in indicates the forward propagation process of the source domain data or the target domain data in the shared feature extractor. It should be noted that the forward propagation processes of the source domain data and the target domain data are consistent, and the difference lies in that only the source domain data participates in the calculation of the classification loss, and in addition, in the calculation of the field measurement loss, the source domain uses the real category label and the target domain uses the predicted category label.

[0109] Exemplarily, the discrimination loss uses binary cross-entropy loss, and the specific formula is as follows:

[0110]

[0111] Where, n s and n t respectively represent the number of simulation full-aperture SAR images and measured full-aperture SAR images in each batch, and n s and n t are both I, and a one-hot encoding form of a real domain label of the i th simulated full-aperture SAR image and the j th measured full-aperture SAR image in each batch of samples, and respectively represent a predicted domain label of the i th simulated full-aperture SAR image and the j th measured full-aperture SAR image in each batch of samples; for example, in the present application, d s = [1, 0] represents a simulated image, d t = [0, 1] represents a measured image.

[0112] For example, the classification loss uses a multi-class cross-entropy loss, and the specific formula is as follows:

[0113]

[0114] wherein, represents a one-hot encoding form of a class label of the i th simulated full-aperture SAR image in each batch of samples, represents a predicted class label of the i th simulated full-aperture SAR image.

[0115] Here, by calculating the domain metric loss, the network can be constrained to align the source domain and the target domain in the feature space class by class.

[0116] For example, the domain metric loss uses a local maximum mean discrepancy (LMMD) loss, and the specific formula is as follows:

[0117]

[0118] wherein, L represents the number of classes of the simulated full-aperture SAR image or the measured full-aperture SAR image in the preset data set, H is a reproducing kernel Hilbert space defined by a kernel function k (·), represents a multi-modal fusion feature of the i th simulated full-aperture SAR image in each batch of samples, represents a multi-modal fusion feature of the j th simulated full-aperture SAR image in each batch of samples, represents a multi-modal fusion feature of the i th measured full-aperture SAR image in each batch of samples, represents a multi-modal fusion feature of the j th measured full-aperture SAR image in each batch of samples, and φ (·) represents a feature mapping of the multi-modal fusion feature to the reproducing kernel Hilbert space. Specifically, the kernel function k (·) uses a Gaussian kernel function with a bandwidth set as an average pairwise squared distance of the source domain and the target domain samples in the feature space; and respectively represent the probability that the i-th simulated full-aperture SAR image and the i-th measured full-aperture SAR image belong to the l-th class, and the calculation formula is: wherein, represents the true one-hot encoded class label vector of the i-th simulated full-aperture SAR image in a batch of samples of the l-th dimension of the i-th simulated full-aperture SAR image in a batch of samples represents the predicted one-hot encoded class label vector of the i-th measured full-aperture SAR image in a batch of samples of the l-th dimension of the i-th measured full-aperture SAR image in a batch of samples

[0119] S25, according to the discriminative loss, the classification loss, the domain metric loss and the weight coefficient of the current training, the total loss of the current training is calculated.

[0120] Here, the total loss is the sum of the discriminative loss, the classification loss and the domain metric loss, and the specific formula is as follows:

[0121] L total =Loss dis +Loss cls +λ·Loss met ;

[0122] wherein, λ is defined as: wherein, epoch current represents the current iteration number, and epoch total represents the total iteration number, and λ gradually increases with the training.

[0123] S26, based on the total loss of the current training, the network parameters in the deep recognition network to be trained at this time are updated through back propagation, and the deep recognition network to be trained next time is obtained, and the iteration is continued until the training times or the convergence degree of the network meet the preset conditions, and the trained deep recognition network is obtained.

[0124] Here, the preset condition can be a preset training time or a preset loss value, which is not limited by the present application.

[0125] The present application also provides a multi-modal feature fusion and global-local domain alignment UDA SAR ATR device, comprising a processor, a communication interface, a memory and a communication bus, the processor, the communication interface and the memory complete the communication among each other through the communication bus; the memory is used to store computer programs; the processor is used to execute the programs stored on the memory, and the steps of the multi-modal fusion feature fusion and global-local alignment UDA SAR ATR method described above are realized.

[0126] ​​Compared with a related method of recognizing a measured full-aperture SAR image based on a template matching idea, the application does not need a large number of template images, and since the application is a method based on a deep neural network, after training is completed, the recognition of images is very efficient. Compared with a related method of introducing a domain adaptation theory and directly inputting an amplitude SAR image into a deep convolutional neural network to extract features for recognition, the application considers different imaging characteristics of SAR images and optical images, introduces topological structure knowledge of a target, and thus improves cross-domain recognition performance. Compared with a related method of combining SAR domain knowledge and deep learning, the application more fully mines imaging characteristics of SAR images, introduces sub-aperture information in visual feature extraction, introduces position information of key points in scattering topological feature extraction, and simultaneously designs a gating feature fusion module, so that the discriminative ability of features is significantly improved, and the discriminative efficiency is high. Compared with a related recognition method based on pseudo-label denoising, the application does not need a denoising process when aligning overall distribution and local distribution between simulation images and measured images, and does not need template matching calculation, so that the training and recognition efficiencies of the application are both high. Compared with a method of treating all samples equally, the application introduces an idea of difficult sample mining, guides the model to focus on difficult samples in global domain alignment, reduces interference of difficult samples on subsequent local domain alignment, and thus more accurately aligns distribution of simulation images and measured images in a feature space, so that cross-domain recognition performance is significantly improved.

[0127] The application adopts a sub-aperture decomposition algorithm to divide a measured full-aperture SAR image into multiple different sub-aperture images, and constructs a scattering topological graph of the measured full-aperture SAR image, then adopts a trained deep recognition network to process the sub-aperture images and the scattering topological graph to obtain the category of the target contained in the measured full-aperture SAR image. The application introduces sub-aperture information of a SAR image into traditional visual feature extraction, and significantly enhances the discrimination ability of visual features to targets through sub-aperture feature fusion; the application introduces position information of key points through position coding when constructing a scattering topological graph of a target in a SAR image, and further combines with the descriptor information of the key points to better model the topological structure of the target, thereby significantly improving the discrimination ability of scattering topological features to targets; the application introduces a gating mechanism when fusing multi-modal features, and adaptively mines complementary information and suppresses conflicting information between different modal features through the gating feature fusion module, thereby significantly improving the fusion effect; the application introduces a gradient-based difficult sample mining idea when aligning the global distribution of the simulation image and the measured image, and further corrects the weight of the difficult sample through power transformation and temperature-based Softmax normalization, thereby reducing the negative impact of the difficult sample on subsequent local domain alignment, and significantly improving the cross-domain recognition performance of the model. In summary, the application significantly improves the cross-domain recognition performance through multi-modal feature fusion and global-local domain alignment, thereby improving the recognition accuracy of the target category in the measured full-aperture SAR image.

[0128] The effect of the target recognition method of the application is further described below through a simulation experiment.

[0129] Experimental conditions:

[0130] The hardware platform of the simulation experiment is: the processor is Intel(R) Xeon(R) Silver 4210R CPU@2.40GHz, the memory is 128GB, and the graphics card is GeForce RTX 4090. The software platform is: Ubuntu 20.04 operating system, python3.8, pytorch 1.12 and matlab 2021a.

[0131] Experimental data: The experimental data used the public SAMPLE dataset, which contains pairs of real and simulated SAR images. This dataset was collected by an X-band HH-polarized radar and contains 10 classes of ground vehicle targets, namely 2S1, BMP2, BTR70, M1, M2, M35, M548, M60, T72 and ZSU23. Each image has a resolution of 0.3 m x 0.3 m, a size of 128 x 128 pixels, an azimuth range of 10°-80°, and an elevation range of 14°-17°. The dataset contains 1345 image pairs, each pair containing one real and one simulated image. There are two experimental scenarios for this dataset. In the first scenario, the source domain data consists of simulated images at 14°-16°, and the target domain data consists of real images at 17°. In the second scenario, the source domain data includes all simulated images, and the target domain data includes all real images. The data details for these two scenarios are shown in Tables 1 and 2, respectively.

[0132] Table 1 Data details for the first scenario

[0133]

[0134]

[0135] Table 2 Data details for the second scenario

[0136]

[0137]

[0138] Comparative experimental method: the results under the same conditions are compared by using the method and various existing algorithms, and the comparison method includes the literature "A SAR dataset for ATR development: The synthetic and measured paired labeled experiment (SAMPLE)" (Proc. SPIE) published by B. Lewis et al. in 2019, the literature "A deep learning approach to the synthetic and measured paired and labeled experiment (SAMPLE) challenge problem" (Proc. SPIE) published by T. Scarnati et al. in 2019, the literature "Bridging a gap in SAR-ATR: Training on fully synthetic and testing on measured data" (IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens.) published by N. Inkawhich et al. in 2019, the literature "Adversarial training on SAR images" (Proc. SPIE) published by B. Lewis et al. in 2020, the literature "A hierarchical receptive network oriented to target recognition in SAR images" (Pattern Recognit.) published by G. Dong et al. in 2022, the literature "Exploring characteristics of neural network architecture computation for enabling SAR ATR" (Proc. SPIE) published by Melzer et al. in 2021, and the literature "Gradual domain adaptation with pseudo-label denoising for SAR target recognition when using only synthetic data for training" (Remote Sens.) published by Sun et al. in 2023.Zhang et al., "VSFA: Visual and scattering topological feature fusion and alignment network for unsupervised domain adaptation in SAR target recognition," IEEE Trans. Geosci. Remote Sens., vol. 61, no. 10, pp. 6578-6593, Oct. 2023.

[0139] Evaluation criteria: SAR image target recognition accuracy.

[0140] Experimental content:

[0141] The above experimental data were identified by the method of the present application and the existing method, and the comparative experimental results of the present application and the existing method under two experimental settings were shown.

[0142] Experimental results and analysis:

[0143] Comparative test:

[0144] Tables 3 and 4 are the identification accuracy of the method of the present application and the existing method under experimental setting 1 and experimental setting 2. As shown in Tables 3 and 4, the method proposed by the present application is significantly better than the existing method, indicating the effectiveness of the method proposed by the present application.

[0145] Table 3 Identification accuracy of each method under experimental setting 1

[0146] Method Accuracy of identification Lewis (2019) 24.97％ Scarnati (2019) 55.62％ Inkawhich (2021) 95.06％ Sun (2023) 97.36％ Zhang (2023) 99.15％ Method proposed by the invention 99.59％

[0147] Table 4 Identification accuracy of each method under experimental setting 2

[0148] Method Accuracy of identification Lewis (2020) 82.05％ Dong (2021) 93.66％ Sun (2023) 96.46％ Melzer (2021) 96.88％ Zhang (2023) 98.18％ Method proposed by the invention 99.37％

[0149] It should be noted that the terms "first", "second" are only for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include one or more features. In the description of the present application, the meaning of "multiple" is two or more, unless otherwise specifically limited.

[0150] In the description of the specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" and the like means that the specific features or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features or characteristics described can be combined in any suitable manner in one or more embodiments or examples. In addition, those skilled in the art can combine and combine different embodiments or examples described in the specification.

[0151] In the specification, the word "comprising" does not exclude other components or steps, and "one" or "an" does not exclude a plurality. Some measures are described in mutually different embodiments, but this does not mean that these measures cannot be combined to produce good results.

[0152] The above is a further detailed description of the present application in combination with specific preferred embodiments, and cannot be considered as limiting the specific implementation of the present application to these descriptions. For those skilled in the art, without departing from the concept of the present application, a number of simple deductions or substitutions can be made, which should be considered as belonging to the protection scope of the present application.

Claims

1. A multi-modal feature fusion and global-local domain alignment UDA SAR ATR method, characterized in that, The method comprises the following steps: acquiring a measured full-aperture SAR image to be identified; using a sub-aperture decomposition algorithm to divide the measured full-aperture SAR image into a group of sub-aperture images; the group of sub-aperture images comprises a plurality of different sub-aperture images; constructing a scattering topology graph of the measured full-aperture SAR image; using a trained deep identification network to process the group of sub-aperture images and the scattering topology graph of the measured full-aperture SAR image to obtain the category of the target contained in the measured full-aperture SAR image; wherein the trained deep identification network is obtained by training an initial deep identification network based on a preset data set; the preset data set comprises a plurality of categories of simulated full-aperture SAR images and a plurality of categories of measured full-aperture SAR images, and each simulated full-aperture SAR image in the preset data set has a real domain label and a real category label, and each measured full-aperture SAR image has a real domain label, wherein the domain label is used to indicate whether the image is a simulated image or a measured image, and the category label is used to indicate the category of the target in the image.

2. The multimodal feature fusion and global-local domain alignment based UDA SAR ATR method according to claim 1, characterized in that, The trained deep identification network comprises a trained sub-aperture feature fusion network, a trained location-aware graph attention network, a trained gating feature fusion module, and a trained classifier; and the processing of the group of sub-aperture images and the scattering topology graph of the measured full-aperture SAR image using the trained deep identification network to obtain the category of the target contained in the measured full-aperture SAR image comprises: using the trained sub-aperture feature fusion network to extract the sub-aperture visual features of each sub-aperture image in the group of sub-aperture images of the measured full-aperture SAR image, and fusing the obtained sub-aperture visual features along the aperture dimension to obtain the visual features of the measured full-aperture SAR image; using the trained location-aware graph attention network to extract the features of each node in the scattering topology graph of the measured full-aperture SAR image, and performing pooling on the extracted features to obtain the scattering topology feature vector of the measured full-aperture SAR image; using the trained gating feature fusion module to fuse the visual features and the scattering topology feature vector through a gating mechanism to obtain the multi-modal fusion features of the measured full-aperture SAR image; using the trained classifier to perform classification processing on the multi-modal fusion features of the measured full-aperture SAR image to obtain the category of the target contained in the measured full-aperture SAR image.

3. The multimodal feature fusion and global-local domain alignment based UDA SAR ATR method according to claim 2, characterized in that, The scattering topology graph comprises a plurality of nodes and initial features of each node; the trained location-aware graph attention network extracts the features of each node in the scattering topology graph of the measured full-aperture SAR image, and performs pooling on the extracted features to obtain the scattering topology feature vector of the measured full-aperture SAR image, which comprises: The trained position-aware graph attention network is used to perform first graph convolution processing on initial features of each node in a scattering topology graph of the measured full-aperture SAR image, and to perform first linear processing on the initial features of each node in the scattering topology graph of the measured full-aperture SAR image, and the obtained features are added to obtain first added features; The first added features are activated to obtain first activated features; The first activated features are subjected to second graph convolution processing and second linear processing, and the obtained features are added to obtain second added features; The second added features are activated to obtain second activated features; The second activated features are pooled to obtain a scattering topology feature vector of the measured full-aperture SAR image.

4. The multimodal feature fusion and global-local domain alignment based UDA SAR ATR method according to claim 2, characterized in that, The trained gating feature fusion module is used to fuse the visual features and the scattering topology feature vector through a gating mechanism to obtain a multi-modal fusion feature of the measured full-aperture SAR image, including: The trained gating feature fusion module is used to perform third linear processing on the visual features to obtain first visual features; The visual features and the first visual features are subjected to Hadamard product to obtain transformed visual features; The scattering topology feature vector is subjected to third linear processing to obtain first scattering topology features; The scattering topology feature vector and the first scattering topology features are subjected to Hadamard product to obtain transformed scattering topology feature vectors; The transformed visual features and the transformed scattering topology feature vectors are spliced, and then fourth linear processing and activation processing are sequentially performed to obtain the multi-modal fusion feature of the measured full-aperture SAR image.

5. The multimodal feature fusion and global-local domain alignment based UDA SAR ATR method according to claim 1, characterized in that, The initial depth recognition network includes: an initial sub-aperture feature fusion network, an initial position-aware graph attention network, an initial gated feature fusion module, an initial discriminator, and an initial classifier. The initial discriminator predicts the domain labels of a set of sub-aperture images and a full-aperture SAR image corresponding to a scattering topology map input to the initial depth recognition network. These domain labels characterize whether the full-aperture SAR image is a simulated image or a measured image. The first layer of the initial discriminator is a power-normalized weighted gradient inversion layer. This layer performs an identity mapping on the input during forward propagation and inverts and multiplies the gradient of the input by a weight coefficient u during backpropagation. i ", the weighting coefficient u i The expression for "" is: in, The domain label of a full-aperture SAR image corresponding to the i-th sub-aperture image and the i-th scattering topology map of the initial depth recognition network, as predicted by the discriminator, u i The information entropy of the predicted domain label of a full-aperture SAR image corresponding to the i-th sub-aperture image and the i-th scattering topology map is represented by α, where α represents the preset power and τ represents the temperature coefficient in softmax normalization. This represents a set of samples in a batch during each network training session, where each sample is a simulated full-aperture SAR image or a measured full-aperture SAR image.

6. The multi-modal fused feature fusion and global-local domain alignment based UDA SAR ATR method according to claim 1, characterized in that, The measured full-aperture SAR image is divided into a group of sub-aperture images by using the sub-aperture decomposition algorithm, including: One-dimensional Fourier transform is performed on the measured full-aperture SAR image along the azimuth direction to convert the measured full-aperture SAR image to the range-Doppler domain to obtain a Doppler domain image; The Doppler domain image is subjected to zero removal operation and window removal operation in sequence to obtain a window-removed image; The spectrum of the window-removed image is equally divided into three parts along the azimuth direction to obtain three sub-spectra, wherein the three sub-spectra have overlapping parts, and each sub-spectrum contains 1 / 2 of the original spectrum of the window-removed image; Each sub-spectrum is subjected to windowing operation, zero padding operation, one-dimensional inverse Fourier transform, quarter power amplitude normalization, and center cropping operation in sequence to obtain a sub-aperture image of a preset size; the three sub-aperture images corresponding to the three sub-spectra constitute a group of sub-aperture images of the measured full-aperture SAR image.

7. The multi-modal fused feature fusion and global-local domain alignment based UDA SAR ATR method according to claim 1, characterized in that, The scattering topology graph of the measured full-aperture SAR image is constructed, including: The measured full-aperture SAR image is subjected to contrast enhancement processing to obtain a processed image; extracting a scattering topology point of the processed image, and generating a descriptor for each extracted scattering topology point; each extracted scattering topology point has a two-dimensional spatial coordinate, which includes a range direction coordinate and an azimuth direction coordinate; for each scattering topology point, generating an initial feature of the scattering topology point according to the two-dimensional spatial coordinate and the descriptor of the scattering topology point; taking the scattering topology point as a node, taking the initial feature of the scattering topology point as the initial feature of the node, and determining a plurality of neighbor nodes of each node according to the Euclidean distance between the nodes, generating a unidirectional edge from each node to each neighbor node of the node, and obtaining a scattering topology graph of the measured full-aperture SAR image.

8. The multimodal feature fusion and global-local domain alignment based UDA SAR ATR method according to claim 7, characterized in that, The two-dimensional spatial coordinate includes a range direction coordinate and an azimuth direction coordinate. The initial feature of the scattering topology point is generated according to the two-dimensional spatial coordinate and the descriptor of the scattering topology point, including: respectively calculating the K-bit position encoding of the range direction coordinate and the K-bit position encoding of the azimuth direction coordinate of the scattering topology point, K being a preset integer; splicing the K-bit position encoding of the range direction coordinate and the K-bit position encoding of the azimuth direction coordinate to obtain the position encoding of the two-dimensional spatial coordinate of the scattering topology point; calculating the average of the descriptor of the scattering topology point and the position encoding of the two-dimensional spatial coordinate of the scattering topology point, and taking the average as the initial feature of the scattering topology point.

9. The multi-modal fused feature fusion and global-local domain alignment based UDA SAR ATR method according to claim 1, characterized in that, The trained deep recognition network is obtained by training in the following method: obtaining the preset data set; using a sub-aperture decomposition algorithm to divide each image in the preset data set into a group of sub-aperture images, and constructing a scattering topology graph of each image in the preset data set, to obtain a group of sub-aperture images and a scattering topology graph of each image in the preset data set; in each training of the deep recognition network, selecting I simulated full-aperture SAR images and I measured full-aperture SAR images from the preset data set as 2I training samples; I is a positive integer greater than 0; inputting the group of sub-aperture images and the scattering topology graph of each training sample in the 2I training samples into the deep recognition network to be trained this time to obtain the output of the discriminator and the output of the classifier in the deep recognition network to be trained this time; according to the output of the gating feature fusion module, the output of the discriminator, the output of the classifier, the value I, the real domain label and the real class label of the I simulated training samples in this training, and the real domain label of the I measured training samples, calculating the discrimination loss, the classification loss and the domain measurement loss of this training; according to the discrimination loss, the classification loss, the domain measurement loss and the weight coefficient of this training, calculating the total loss of this training; based on the total loss of this training, performing back propagation to update the network parameters in the deep recognition network to be trained this time, obtaining the deep recognition network to be trained next time, and iterating until the training times or the convergence degree of the network meets the preset condition, obtaining the trained deep recognition network.

10. A multi-modal fusion feature fusion and global-local domain alignment UDA SAR ATR device, comprising a processor, a communication interface, a memory and a communication bus, characterized in that, The processor, the communication interface and the memory complete the communication with each other through the communication bus; The memory is used for storing computer programs; The processor is used for executing the programs stored on the memory to realize the method steps in any one of claims 1-9.

Images