SAR ship target classification method based on multi-scale cross-polarization attention network

Abstract

The application provides a SAR ship target classification method based on a multi-scale cross-polarization attention network, and relates to the technical field of radar target recognition.The application solves the problems of less available information, speckle noise, trailing noise interference and insufficient use of polarization information in the SAR image, and the application uses FBC for fusion, which can achieve better results compared with grouped bilinear pooling; and the use of multi-scale information can improve the disadvantage of less SAR ship information and fully utilize the context information; compared with channel attention and spatial attention, cross-polarization attention is more suitable for SAR ship classification, can guide each other between the two polarizations, and fully utilizes the complementary information between each other; and the network training of the adaptive loss weight adjustment is more balanced, which improves the effectiveness and classification performance of the network framework for SAR ship target feature extraction.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of radar target recognition, and particularly relates to a SAR ship target classification method based on a multi-scale cross-polarization attention network. BACKGROUND

[0002] In the past few decades, the launch of several advanced synthetic aperture radar satellites has driven the development of the SAR remote sensing field. Ship classification using SAR images is a more advanced and challenging task. Early work designed traditional hand-crafted features, derived from geometric and scattering properties, and was mostly evaluated on limited high-resolution SAR ship samples, and its performance would be significantly reduced if applied to medium-resolution (MR) SAR images. In addition, large-scale SAR ship data poses a challenge to the above features. Researchers in the SAR remote sensing field have made efforts to apply deep learning techniques to obtain more discriminative SAR ship representations.

[0003] Considering the polarization information in SAR images, some works have attempted to use dual-polarization information to obtain better performance. Xi et al. proposed a new feature loss dual fusion network (DFSN), which uses a target detection network to extract the ship part, then uses a twin network to extract features, and finally uses a multi-loss to jointly monitor network learning. Zeng et al. developed a hybrid channel feature loss (HCFL) for deep CNN features to jointly utilize the information contained in the SAR images of VH and VV. Zhang et al. proposed a dual-polarization feature fusion (SE-LPN-DPFF) compressed excitation Laplacian pyramid network for SAR ship classification. He et al. improved the bilinear pooling and proposed a group bilinear pooling network (GBCNN).

[0004] In summary, the above methods have some defects: first, the DFSN network is not an end-to-end network, and it needs to extract the ship part in the image first, which is time-consuming and laborious compared to other networks. Second, the above processing methods only use the features of the last layer for processing, and the last layer of convolutional features can represent the most discriminative semantic features, but they do not have semantic integrity, and in SAR ship classification, any detailed information can be the key to improving accuracy. Third, when fusing two polarization information, simply using a first-order fusion method (e.g., concatenation, summation or convolution) cannot fully utilize the complementary information of the two polarizations. However, when using a second-order bilinear pooling operation, the features will contain redundant information, speckle noise, sidelobes and tail effects, which limits the performance. Therefore, how to fully utilize the polarization information and suppress the influence of redundant information and noise in the utilization process is very crucial. Therefore, there is still room for improvement for SAR ship classification. SUMMARY

[0005] This invention addresses the problems of limited usable information, speckle noise, trailing noise, and insufficient utilization of polarization information in existing SAR images. It provides a SAR ship target classification method based on a multi-scale cross-polarization attention network, characterized by low computational cost, high training efficiency, and accurate classification. This invention is tailored to the imaging characteristics of SAR ships, utilizing rich multi-scale information and improving the attention mechanism. It also employs adaptive loss optimization to improve the network structure. By fully utilizing the features of different network stages, it acquires rich contextual information. Furthermore, it proposes a cross-polarization attention structure, fully leveraging the complementary information of the two polarizations, allowing them to guide each other. Compared to the original attention mechanism, this is more suitable for the SAR ship domain. Finally, it utilizes an adaptive learning weight method to adaptively adjust the weights of each loss term, thereby balancing each loss term in the network and achieving superior ship classification.

[0006] This invention provides a SAR ship target classification method based on a multi-scale cross-polarity attention network, which includes the following steps:

[0007] Step 1: Select two types of radar ship images from the OpenSARShip dataset: VV polarization (vertical transmit / vertical receive signal) and VH polarization (vertical transmit / horizontal receive signal). Divide the VV and VH polarization ship images into a training set Φ at an 8:2 ratio. x and test set Φ c ; the training set Φ x The training set Φ was modified using methods such as flipping, rotating, translating, and adding noise. x The training set Φ is augmented by a factor of k to obtain the augmented training set Φ. x ′, where k≥2, and k is a natural number.

[0008] Step 2: Construct a multi-scale cross-polarity attention network Ω, which is composed of a deep dense connection layer, a multi-scale feature fusion layer, a cross-polarity attention layer, and an adaptive learning weight layer connected in series.

[0009] Step 3 trains a multi-scale cross-polarity attention network Ω, wherein the multi-scale cross-polarity attention network Ω is trained by obtaining a target loss function L that adaptively learns weights. ALL To train the network, the expanded training set Φ x The input is fed into the constructed multi-scale transpolar attention network Ω, and the output is a classification result, which is the probability that each sample is predicted to belong to each category. The category with the highest probability is selected as the predicted category of the sample. The pre-constructed target loss function L is used. ALL A multi-scale, cross-polarity attention network is trained until the target loss function L is reached. ALLConverge to obtain the trained multi-scale cross-polarization attention network Ω';

[0010] Step 4, the test set Φ c is input into the trained multi-scale cross-polarization attention network Ω', to obtain the predicted classification result of the radar ship image, and the predicted classification result of the radar ship image is compared with the true class of the test set Φ c , and according to the calculation method of the evaluation index, the final target evaluation index is obtained, and the final target evaluation index is compared with the evaluation index of other SAR ship classification methods.

[0011] Preferably, in step 1, two radar ship images of vertical transmission vertical reception signal VV polarization and vertical transmission horizontal reception signal VH polarization are selected from the OpenSARShip dataset, and the two polarized ship images of VV polarization and VH polarization are selected in an 8:2 ratio to divide into a training set Φ x and a test set Φ c ; the training set Φ x is obtained as follows:

[0012] Step 101 finds the ship data from the obtained OpenSARShip data file, and selects the target SAR (Synthetic Aperture Radar) image corresponding to the data of the ship type of tanker, container ship and bulk carrier for calibration;

[0013] Step 102, according to the class and position information of the ship slice in the OpenSARShip dataset, re-crops the calibrated target SAR image into a 64x64 ship slice;

[0014] Step 103 replaces the redundant ship slices in the cropped slice of the target SAR image with a background, and each slice has only one ship target, to obtain the training set Φ x .

[0015] Preferably, in step 1, the training set Φ x is expanded k times by using the methods of flipping, rotating, shifting and adding noise to obtain the expanded training set Φ x ', and the specific steps include the following:

[0016] Step 1031 horizontally flips and flips up and down the training set Φ x , respectively, to obtain a 2-fold expanded training set;

[0017] Step 1032 rotates the training set Φ x by 90°, 180° and 270° clockwise, respectively, to obtain a 3-fold expanded training set;

[0018] Step 1033 translates the ship target in the training set Φ x by any pixel point in the up-down and left-right four directions [-5, 5], and obtains an expanded 1-time training set;

[0019] Step 1034 adds Gaussian noise with mean 0 and variance 0.001 to the training set Φ x , and obtains an expanded 1-time training set;

[0020] Step 1035 combines the training set obtained after steps 104 to 107 and the original training set Φ x , and forms a final expanded 8-time training set Φ x .

[0021] Preferably, the multi-scale cross-polarization attention network Ω in step 2 is composed of a deep dense connection layer, a multi-scale feature fusion layer, a cross-polarization attention layer, and an adaptive learning weight layer in series; The specific steps are as follows:

[0022] Step 201 constructs a deep dense connection layer, which is composed of two identical branches, and inputs two polarized radar images at the same time, each branch is composed of a basic layer S and a dense connection layer D1, a transition dimension reduction layer T1, a dense connection layer D2, a transition dimension reduction layer T2, a dense connection layer D3, a transition dimension reduction layer T3, a dense connection layer D4, a transition dimension reduction layer T4, and a dense connection layer D5 in series, and each layer of the deep dense connection layer is extracted;

[0023] Step 202 constructs a multi-scale feature fusion layer, which is composed of m down-sampling layers DS m , 1 up-sampling layer U1, a feature fusion layer FBC, and a 1×1 convolution kernel C1 in series; The multi-scale feature fusion is performed on the feature maps of T3, T4 and D5 respectively, and then the features of the two polarizations are fused to obtain a first bilinear vector v ms ;

[0024] Step 203 constructs a cross-polarization attention layer, which is composed of a first-order attention layer F SE and a second-order attention layer S SE in series, and the multi-scale fusion results F vh and F vv of the two radar ship images on the dense connection layer D5 are obtained, the multi-scale fusion result F vh is sent to the first-order attention layer F SE to obtain a cross-polarization attention vector v vh , and the multi-scale fusion result F vv is sent to the second-order attention layer S SE to obtain a cross-polarization attention vector vvv The transpolar attention vector v vh Multiply by the multi-scale fusion result F vv Obtain the transpolarization attention feature F vv ′, will the transpolarization attention vector v vv Multiply by the multi-scale fusion result F vh Obtain the transpolarization attention feature F vh Then, the transpolarization attention feature F vv ′ and transpolarization attention feature F vh The vector is fed into the feature fusion layer FBC to obtain the second bilinear vector v. vh-vv Simultaneously, the transpolarity attention feature F vh The vector is fed into the feature fusion layer FBC to obtain the third bilinear vector v. vh-vh Transpolar attention feature F vv The vector is fed into the feature fusion layer FBC to obtain the fourth bilinear vector v. vv-vv ;

[0025] Step 204: Construct an adaptive learning loss weight layer, wherein the adaptive learning loss weight layer will convert the first bilinear vector v ms The second bilinear vector and the third bilinear vector v vh-vh The fourth bilinear vector v vv-vv After passing through the Softmax classifier layer, the cross-entropy loss L of the first bilinear vector is obtained. Mutil_scale The cross-entropy loss L of the second bilinear vector VH-VH The cross-entropy loss L of the third bilinear vector VV-VV The cross-entropy loss L of the fourth bilinear vector VH-VV To obtain a new loss function L for adaptive learning weights. ALL To train the network, the loss function L is determined. ALL for:

[0026] L ALL =w1L Mutil_scale +w2L VH-VH +w3L VV-VV +w4L VH-VV (1)

[0027] In the formula, L Mutil_scale L represents the cross-entropy loss of the multi-scale feature fusion term. VH-VH L represents the cross-entropy loss of a vertically transmitted, horizontally received VH-polarized radar image. VV-VV L represents the cross-entropy loss of a vertically transmitted, vertically received VV-polarized radar image. VH-VVCross-entropy loss of vertical transmission horizontal reception VH polarized radar image and vertical transmission vertical reception VV polarized radar image, w1, w2, w3 and w4 represent the weight of each loss.

[0028] Preferably, the base layer S in step 201 is composed of two identical convolution blocks in series, each of which is composed of a convolution layer L1, a batch normalization layer L2, and an activation function layer L3 in series, and the activation function layer L3 outputs a nonlinear mapping ReLU(x) as follows:

[0029] ReLU(x)=max(0,x) (2)

[0030] In formula (2), x is the output of the batch normalization L2 layer, the convolution kernel size is 3, and the step is 1;

[0031] The dense connection layers D1, D2, D3, D4 and D5 are composed of three identical convolution blocks in series according to their respective growth rates, each of which is composed of a batch normalization layer an activation function layer a convolution layer a dropout layer in series, respectively obtaining the corresponding dense connection layer convolution kernel size and step size; wherein i∈{1,2,3,4,5}, n∈{1,2,3}, n represents the nth convolution block;

[0032] The transition dimension reduction layers T1, T2, T3 and T4 are composed of a batch normalization layer an activation function layer a convolution layer an average pooling layer in series, respectively obtaining the corresponding transition dimension reduction layer convolution kernel size and step size, according to the transition dimension reduction layer down-sampling kernel window size, obtaining the corresponding transition dimension reduction layer step size, wherein i∈[1,2,3,4].

[0033] Preferably, the multi-scale feature fusion layer in step 202 is composed of m down-sampling layers DS m , 1 up-sampling layer U1, one feature fusion layer FBC and one 1×1 convolution layer C1 in series; wherein m≥1, m is a natural number;

[0034] The down-sampling kernel window size and step size of the down-sampling layer DS m are set according to the size of the feature map;

[0035] The up-sampling kernel window size and step size of the up-sampling layer U1 are set according to the size of the feature map;

[0036] The feature fusion layer FBC consists of two parallel fully connected layers f1 and f2, followed by a max-pooling layer M. The size of the pooling kernel window of the max-pooling layer M is set according to the size of the feature map.

[0037] Convolutional layer C1 has a kernel size of 1×1 and a stride of 1;

[0038] Step 202 involves performing multi-scale feature fusion processing on the feature maps of T3, T4, and D5 respectively, and then fusing the features of the two polarizations to obtain four bilinear vectors. Specifically, multi-scale feature fusion processing is first performed on the feature maps of the transition dimensionality reduction layer T3, transition dimensionality reduction layer T4, and dense connection layer D5; then the features of the transition dimensionality reduction layer T3 and transition dimensionality reduction layer T4 are fused, and finally the features of the two radar ship images are fused to obtain the first bilinear vector v. ms .

[0039] The feature fusion layer FBC consists of two parallel fully connected layers f1 and f2, followed by a max-pooling layer M. The pooling kernel window size of the max-pooling layer M is 4. Features of the two polarizations located in the deep densely connected layers at the transition dimensionality reduction layers T1, T2, T3, T4 and D5 are extracted respectively. , , , , , , , , , ;

[0040] T3 features in the transitional dimensionality reduction layer of VV polarization The VV polarization features located in the transition dimensionality reduction layers T1, T2, T4, and D5 are described above. , , and Samples were obtained through downsampling layer DS2, downsampling layer DS3, and upsampling layer U1, respectively. At the same size, they are then added along the channel dimension and merged using a 1×1 convolution kernel C1 to obtain the feature F of the VV-polarized transition dimensionality reduction layer T3 multi-scale feature fusion. vv_3 The specific formula is as follows:

[0041] F vv_3 =C1(DS2(F) vv_1 )+DS3(F vv_2 )+F vv_3+U1(F vv_4 )+U1(F vv_5 (3)

[0042] T3 characteristics of VH polarization The VH polarization features located in the transition dimensionality reduction layers T1, T2, T4, and D5 are described above. , , and Samples were obtained through downsampling layer DS2, downsampling layer DS3, and upsampling layer U1, respectively. At the same size, they are then added along the channel dimension and merged using a 1×1 convolution kernel C1 to obtain the feature F of the VH-polarized transition dimensionality reduction layer T3 multi-scale feature fusion. vh_3 The specific formula is as follows:

[0043] F vh_3 =C1(DS2(F) vh_1 )+DS3(F vh_2 )+F vh_3 +U1( Fvh_4 )+U1(F vh_5 (4)

[0044] Similarly, in terms of features The VV polarization is located in the transition dimensionality reduction layers T1, T2, T4, T3, and D5 of the transition dimensionality reduction layer. , , and Samples were obtained through downsampling layers DS1, DS2, and DS3 respectively. At the same size, they are then added along the channel dimension and merged using a 1×1 convolution kernel C1 to obtain the feature F of the VV-polarized transition dimensionality reduction layer T4 multi-scale feature fusion. vv_4 The specific formula is as follows:

[0045] F vv_4 =C1(DS1(F) vv_1 )+DS2(F vv_2 )+DS3(F vv_3 )+F vv_4 +F vv_ 5) (5)

[0046] T4 features in the transitional dimensionality reduction layer of VH polarization The VH polarization features located in the transition dimensionality reduction layers T1, T2, T3, and D5 are described above. , , and Samples were obtained through downsampling layers DS1, DS2, and DS3 respectively. At the same size, they are then added along the channel dimension and merged using a 1×1 convolution kernel C1 to obtain the feature F of the VH-polarized transition dimensionality reduction layer T4 multi-scale feature fusion. vh_4 The specific formula is as follows:

[0047] F vh_4 =C1(DS1(F) vh_1 )+DS2(F vh_2 )+DS3(F vh_3 )+F vh_4 +F vh_5 (6)

[0048] Features of D5 densely connected layer in VV polarization The VV polarization features located in the transition dimensionality reduction layers T1, T2, T3, and dense connection layer T4 are described above. , , and Samples were obtained through downsampling layers DS1, DS2, and DS3 respectively. At the same size, they are then added along the channel dimension and merged using a 1×1 convolution kernel C1 to obtain the VV-polarized densely connected layer D5, which fuses multi-scale features F. vv_5 The specific formula is as follows:

[0049] F vh_5 =C1(DS1(F) vv_1 )+DS2(F vv_2 )+DS3(F vv_3 )+F vv_4 +F vv_5 (7)

[0050] Features of D5 densely connected layer in VH polarization The VH polarization features located in the transition dimensionality reduction layers T1, T2, T3, and dense connection layer T4 are described above. , , and Samples were obtained through downsampling layers DS1, DS2, and DS3 respectively. At the same size, they are then added along the channel dimension and merged using a 1×1 convolution kernel C1 to obtain the feature F of the VH-polarized densely connected layer D5, which is a multi-scale feature fusion layer. vh_5 The specific formula is as follows:

[0051] F vh_5 =C1(DS1(F) vh_1 )+DS2(F vh_2 )+DS3(F vh_3 )+F vh_4 +F vh_5 (8)

[0052] In VV polarization, the feature F of the transition dimensionality reduction layer T3 and the transition dimensionality reduction layer T4 after multi-scale feature fusion. vv_3 ′ and F vv_4 ′Up, F vv_3 'Sampling is performed at F through the downsampling layer DS3.' vv_4 At the same size, they are then added along the channel dimension and merged using a 1×1 convolution kernel C1 to obtain the VV polarization global multi-scale features C. vv_ms The specific formula is as follows:

[0053] c vv_ms =C1(DS3(F) vv_3 ′)+F vv_4 ′) (9)

[0054] Similarly, regarding VH polarization, the feature F of the transition dimensionality reduction layer T3 and the transition dimensionality reduction layer T4 after multi-scale feature fusion... vh_3 ′ and F vh_4 ′Up, F vh_3 'Sampling is performed at F through the downsampling layer DS3.' vh_4 At the same size, they are then added along the channel dimension and merged using a 1×1 convolution kernel C1 to obtain the VH polarization global multi-scale features C. vh_ms The specific formula is as follows:

[0055] C vh_ms =C1(DS3(F) vh_3 ′)+F vh_4 ′) (10)

[0056] Finally, a feature fusion layer (FBC) is used to feed the result of fusing the two polarization multi-scale methods into two parallel fully connected layers (f1 and f2). The result from the fully connected layers is then multiplied by a dot product and fed into a max pooling layer (M) to obtain the first bilinear vector v. ms The step of obtaining the first bilinear vector v ms As shown in the following formula:

[0057] v ms =M(f1(C vv_ms ′)*f2(C vh_ms ′)) (11)

[0058] Among them, Cvv_ms and C vh_ms represent two parallel fully connected layers, * represents the dot product of two vectors, and M represents the max-pooling layer.

[0059] Preferably, the method of constructing each layer of the cross-polarization attention layer in step 203 is as follows:

[0060] The first-order attention layer F SE is composed of a global average pooling layer, a fully connected layer, a BN layer, a Relu layer, a fully connected layer, and a Sigmoid layer in series.

[0061] The second-order attention layer S SE is composed of a grouped bilinear pooling layer, a fully connected layer, a BN layer, a Relu layer, a fully connected layer, and a Sigmoid layer.

[0062] The two polarizations of the features F vh_5 and F vv_5 obtained in step 202 are located at D5, which are respectively passed through the first-order attention layer F SE and the second-order attention layer S SE to obtain the first-order attention vector v vh and the second-order attention vector v vv The first-order attention vector v vh is multiplied by F vv_5 , and the second-order attention vector v vv is multiplied by F vh_5 to obtain the multiplied features C vh and C vv , the specific formula is as follows, which constitutes the cross-polarization attention layer.

[0063] C vh ′=F vh_5 ′×v vv (12)

[0064] C vv ′=F vv_5 ′×v vh (13)

[0065] The multiplied features C vh and C vv are sent to the feature fusion layer FBC to obtain the second bilinear vector v vh-vv is as follows:

[0066] v vh-vv =M(f1(C vh ′)*f2(C vv ′));

[0067] Meanwhile, the two polarizations of the feature F vv and F vh after multi-scale feature fusion are respectively subjected to single polarization feature fusion FBC operation to obtain a third bilinear vector v vh-vh and a fourth bilinear vector v vv-vv , and the specific formula is as shown in the following formula:

[0068] v vh-vh = M(f1(C vh ′)*f2(C vh ′)) (14)

[0069] v vv-vv = M(f1(C vv ′)*f2(C vv ′)) (15).

[0070] Preferably, the steps of constructing the adaptive learning loss weight layer in step 204 are as follows:

[0071] Step 2041, the first bilinear vector v ms , the second bilinear vector v vh-vv , the third bilinear vector v vh-vh and the fourth bilinear vector v vv-vv are sent into the Softmax classifier layer, and the cross-entropy of each loss is calculated, and the results are respectively the cross-entropy loss L Mutil_scale of the first bilinear vector, the cross-entropy loss L VH-VV of the second bilinear vector based on the multi-scale cross-polarization attention network, the cross-entropy loss L VH-VH of the third bilinear vector and the cross-entropy loss L VV-VV of the fourth bilinear vector.

[0072] Step 2042 uses the gradient norm Gradnorm in the adaptive learning weight method to construct the loss function L grad to learn the weight of each loss, and calculates the l2 norm of the gradient of each loss to calculate the mean of the gradient norm of each loss , and the counter-training rate of each loss at the current iteration is calculated based on the 0th iteration as a benchmark , and finally the relative counter-training rate r i (t) of each loss is calculated according to the , to obtain the fast and slow degree of training of each loss compared to all losses; the specific calculation formula is as shown in the following formula:

[0073] The loss function for updating the weight value of each loss is:

[0074] (16)

[0075] wherein L grad is the l1 norm of the difference between the gradient norm of the i-th loss multiplied by the relative inverse training rate of the i-th loss and the mean of the gradient norms of all losses, i is the number of each loss, i∈{1,2,3,4};

[0076] The gradient norm of each loss is calculated as: (17)

[0077] wherein represents the partial derivative of the i-th loss multiplied by its weight with respect to the parameter w; is the l2 norm of the gradient of the i-th loss multiplied by the i-th weight with respect to the parameter w;

[0078] The mean of the gradient norms is calculated as: (18)

[0079] wherein is the mean of the gradient norms of each loss;

[0080] The inverse training rate of each loss at the current iteration is calculated as: (19)

[0081] wherein L i (0) is the loss of the i-th loss at the 0th iteration number;

[0082] The relative inverse training rate of each loss is calculated as: (20)

[0083] wherein a is a hyperparameter, a∈[0,2), is a balance coefficient, r i (t) is the inverse training rate of each loss at the current iteration divided by the mean of the inverse training rates of all losses at the current iteration .

[0084] Preferably, in step 3, the constructed loss function L ALL is used to train the multi-scale cross-polarization attention network, and the training is performed until the loss function L ALL converges, to obtain a trained multi-scale cross-polarization attention network Ω′, which is specifically referred to as the overall loss function L ALL :

[0085] L ALL = w1L Mutil_scale + w2L VH-VV + w3L VH-VH + w4L VV-VV (1)

[0086] The multi-scale cross-polarization attention network is trained, and the training process is as follows: according to the value of the overall loss function L ALL , back propagation is performed, the gradient is obtained according to the chain rule, and the gradient is used to update all parameters in the network; at the same time, the loss function L grad of the weight value of each loss is also used for back propagation to update the weight wi of each loss, i∈{1,2,3,4};

[0087] Each randomly divided augmented training set Φ x ' is used to train the network Ω for at least three times, and after the trained network Ω' is obtained, the test set Φ c is used for testing, and the median of each index is recorded.

[0088] Preferably, the test set Φ c is input into the trained multi-scale cross-polarization attention network Ω' in step 4 to obtain the predicted classification result of the radar ship image, the predicted classification result is compared with the true class of the test set Φ c , the final target evaluation index is obtained according to the calculation method of the evaluation index, the final target evaluation index is compared with the evaluation index of other SAR ship classification methods, specifically, the predicted classification result is the probability of each sample being predicted as each class, the class with the maximum probability is selected as the predicted class of the sample, and the classification result is compared with the true class of the test set Φ c ; the comparison result is the overall accuracy OA, the precision P, the recall R and the metric F1, and the comparison is made with the related method of SAR ship classification.

[0089] Among them, the overall accuracy OA is the number of correctly predicted samples / total sample number, the precision P is the number of correctly predicted positive examples / total predicted positive example number, the recall R is the number of correctly predicted positive examples / total actual positive example number, and the metric F1 is the harmonic mean of the precision P and the recall R.

[0090] The application provides a SAR ship target classification method based on a multi-scale cross-polarization attention network, solves the problems of less available information, speckle noise, trailing noise interference and insufficient use of polarization information in a SAR image, fuses by using FBC, and compared with grouped bilinear pooling, can achieve better effect; and by using multi-scale information, the disadvantage of less SAR ship information can be improved, and context information is fully used; compared with channel attention and spatial attention, cross-polarization attention is more suitable for SAR ship classification, can guide between the two polarizations, and fully uses complementary information between each other; and adaptive adjustment of the loss weight makes the network training more balanced, is not affected by a certain loss, and improves the effectiveness and classification performance of the network framework for SAR ship target feature extraction. BRIEF DESCRIPTION OF DRAWINGS

[0091] Figure 1 It is a general flowchart for implementation of the SAR ship target classification method based on the multi-scale cross-polarization attention network.

[0092] Figure 2 It is a structural schematic diagram of the multi-scale cross-polarization attention network of the SAR ship target classification method based on the multi-scale cross-polarization attention network.

[0093] Figure 3 It is a structural schematic diagram of the multi-scale feature fusion layer of the SAR ship target classification method based on the multi-scale cross-polarization attention network, taking feature map T3 as an example.

[0094] Figure 4 It is a structural schematic diagram of the cross-polarization attention layer of the SAR ship target classification method based on the multi-scale cross-polarization attention network.

[0095] Figure 5 It is a structural schematic diagram of the FBC feature fusion layer of the SAR ship target classification method based on the multi-scale cross-polarization attention network. DETAILED DESCRIPTION

[0096] The application will be further described in detail below in combination with the drawings and embodiments, but the application is not limited to the following embodiments.

[0097] DETAILED DESCRIPTION

[0098] The SAR ship target classification method based on the multi-scale cross-polarization attention network provided in the embodiment of the application, the method comprises the following steps:

[0099] Step 1: two radar ship images of vertically transmitting vertically receiving signals VV polarization and vertically transmitting horizontally receiving signals VH polarization are selected from an OpenSARShip data set, and the two polarization ship images of VV polarization and VH polarization are selected and divided into a training set Φx and test set Φ c ; the training set Φ x is divided into training set Φ x and test set Φ x ′, wherein k≥2, k is a natural number;

[0100] Step 2, a multi-scale cross-polarization attention network Ω is constructed, which is composed of a deep dense connection layer, a multi-scale feature fusion layer, a cross-polarization attention layer and an adaptive learning weight layer in series;

[0101] Step 3, the multi-scale cross-polarization attention network Ω is trained by obtaining the target loss function L ALL of adaptive learning weight to train the network, and the expanded training set Φ x ′ is input into the constructed multi-scale cross-polarization attention network Ω, and the classification result is output, which is the probability of each sample being predicted as each class. The class with the maximum probability is selected as the predicted class of the sample; the constructed target loss function L ALL is used to train the multi-scale cross-polarization attention network, and the training is stopped until the target loss function L ALL converges, and the trained multi-scale cross-polarization attention network Ω′ is obtained;

[0102] Step 4, the test set Φ c is input into the trained multi-scale cross-polarization attention network Ω′, and the predicted classification result of the radar ship image is obtained. The predicted classification result of the radar ship image is compared with the true class of the test set Φ c , and the final target evaluation index is obtained according to the calculation method of the evaluation index. The final target evaluation index is compared with the evaluation index of other SAR ship classification methods.

[0103] In one embodiment, two radar ship images of vertical transmitting vertical receiving signal VV polarization and vertical transmitting horizontal receiving signal VH polarization are selected from the OpenSARShip dataset in step 1, and the two polarized ship images of VV polarization VH polarization are selected according to the ratio of 8:2. The training set Φ x and test set Φ c are divided; the training set Φ x is obtained as follows:

[0104] Step 101, find the ship data from the obtained OpenSARShip data file, and select the target SAR image corresponding to the data of the ship type of oil tanker, container ship and bulk cargo ship for calibration;

[0105] Step 102, according to the category and position information of the ship slice in the OpenSARShip dataset, the target SAR image is recut into a 64x64 ship slice on the calibrated target SAR image;

[0106] Step 103, the redundant ship slices in the target SAR image cut slice are replaced with the background, and there is only one ship target in a slice, to obtain the training set Φ x .

[0107] In an embodiment, the training set Φ x is expanded k times using the methods of flipping, rotating, translating, and adding noise to obtain the expanded training set Φ x ′, the specific steps include the following:

[0108] Step 1031, the training set Φ x is respectively flipped horizontally and upside down to obtain a 2-fold expanded training set;

[0109] Step 1032, the training set Φ x is respectively rotated by 90°, 180°, and 270° clockwise to obtain a 3-fold expanded training set;

[0110] Step 1033, the ship target in the training set Φ x is randomly translated by [-5, 5] pixel points in the up, down, left, and right directions to obtain a 1-fold expanded training set;

[0111] Step 1034, Gaussian noise with a mean of 0 and a variance of 0.001 is added to the training set Φ x to obtain a 1-fold expanded training set;

[0112] Step 1035, the training sets obtained after steps 104 to 107 are expanded and combined with the original training set Φ x to form a final 8-fold expanded training set Φ x ′.

[0113] In an embodiment, the multi-scale cross-polarization attention network Ω in step 2 is composed of a deep dense connection layer, a multi-scale feature fusion layer, a cross-polarization attention layer, and a self-adaptive learning weight layer in series; the specific steps are as follows:

[0114] Step 201 constructs a deep dense connection layer, the deep dense connection layer is composed of two same branches, two kinds of polarized radar images are input at the same time, each branch is composed of a basic layer S and a dense connection layer D1, a transition dimension reduction layer T1, a dense connection layer D2, a transition dimension reduction layer T2, a dense connection layer D3, a transition dimension reduction layer T3, a dense connection layer D4, a transition dimension reduction layer T4, a dense connection layer D5 are connected in turn, and each layer feature of the deep dense connection layer is extracted;

[0115] Step 202 constructs a multi-scale feature fusion layer, the multi-scale feature fusion layer is composed of m down-sampling layers DS m , 1 up-sampling layer U1, a feature fusion layer FBC and a 1x1 convolution kernel C1 in turn; multi-scale feature fusion processing is carried out on the feature maps of T3, T4 and D5 respectively, then the features of two kinds of polarization are fused to obtain a first bilinear vector v ms ;

[0116] Step 203 constructs a cross-polarization attention layer, the cross-polarization attention layer is composed of a first-order attention layer F SE , a second-order attention layer S SE in turn, multi-scale fusion is carried out on two kinds of radar ship images located on the dense connection layer D5 to obtain multi-scale fusion results F vh and F vv , the multi-scale fusion result F vh is sent to the first-order attention layer F SE to obtain a cross-polarization attention vector v vh , the multi-scale fusion result F vv is sent to the second-order attention layer S SE to obtain a cross-polarization attention vector v vv , the cross-polarization attention vector v vh is multiplied by the multi-scale fusion result F vv to obtain a cross-polarization attention feature F vv ', the cross-polarization attention vector v vv is multiplied by the multi-scale fusion result F vh to obtain a cross-polarization attention feature F vh ', then the cross-polarization attention feature F vv ' and the cross-polarization attention feature F vh ' are sent to the feature fusion layer FBC to obtain a second bilinear vector v vh-vv ; at the same time, the cross-polarization attention feature F vh ' is sent to the feature fusion layer FBC to obtain a third bilinear vector v vh-vh , and the cross-polarization attention feature F vv ' is sent to the feature fusion layer FBC to obtain a fourth bilinear vector v vv-vv ;

[0117] Step 204: Construct an adaptive learning loss weight layer, wherein the adaptive learning loss weight layer will convert the first bilinear vector v ms The second bilinear vector and the third bilinear vector v vh-vh The fourth bilinear vector v vv-vv The cross-entropy loss L of the first bilinear vector obtained after passing through the Softmax classifier layer. Mutil_scale The cross-entropy loss L of the second bilinear vector VH-VH The cross-entropy loss L of the third bilinear vector VV-VV The cross-entropy loss L of the fourth bilinear vector VH-VV To obtain a new loss function L for adaptive learning weights. ALL To train the network, the loss function L is determined. ALL for:

[0118] L ALL =w1L Mutil_scale +w2L VH-VH +w3L VV-VV +w4L VH-VV (1)

[0119] In the formula, L Mutil_scale L represents the cross-entropy loss of the multi-scale feature fusion term. VH-VH L represents the cross-entropy loss of a vertically transmitted, horizontally received VH-polarized radar image. VV-VV L represents the cross-entropy loss of a vertically transmitted, vertically received VV-polarized radar image. VH-VV denoted as , representing the cross-entropy loss of vertically transmitted horizontally received VH-polarized radar images and vertically transmitted vertically received VV-polarized radar images, where w1, w2, w3, and w4 represent the weights of each loss term.

[0120] In one embodiment, the base layer S in step 201 consists of two identical convolutional blocks connected in series. Each convolutional block consists of a convolutional layer L1, a batch normalization layer L2, and an activation function layer L3 connected in series. The activation function layer L3 outputs the nonlinear mapping ReLU(x) as follows:

[0121] ReLU(x) = max(0,x) (2)

[0122] In equation (2), x is the output of the batch normalized L2 layer, the kernel size is 3, and the stride is 1.

[0123] Densely connected layers D1, D2, D3, D4, and D5 are composed of three identical convolutional blocks strung together, each with a batch normalization layer, according to their respective growth rates. Activation function layer Convolutional layer dropout layer are sequentially connected in series, respectively obtaining corresponding dense connection layer convolution kernel size and step size; wherein, i∈{1, 2, 3, 4, 5}, n∈{1, 2, 3}, n represents the nth convolution block;

[0124] The transition dimension reduction layers T1, T2, T3 and T4 are composed of batch normalization layers , activation function layers convolution layers average pooling layers are sequentially connected in series, respectively obtaining corresponding transition dimension reduction layer convolution kernel size and step size, and the step size of the corresponding transition dimension reduction layer is obtained according to the transition dimension reduction layer down-sampling kernel window size, wherein, i∈[1, 2, 3, 4].

[0125] In one embodiment, the multi-scale feature fusion layer in step 202 is sequentially composed of m down-sampling layers DS m , 1 up-sampling layer U1, one feature fusion layer FBC and one 1×1 convolution layer C1; wherein, m≥1, m is a natural number;

[0126] The down-sampling kernel window size and step size of the down-sampling layer DS m are set according to the size of the feature map;

[0127] The up-sampling kernel window size and step size of the up-sampling layer U1 are set according to the size of the feature map;

[0128] The feature fusion layer FBC is composed of two parallel full connection layers f1 and f2, and then a Max-Pooling maximum pooling layer M is connected in series, and the pooling kernel window size of the maximum pooling layer M is set according to the size of the feature map;

[0129] The convolution layer C1 has a convolution kernel size of 1×1 and a step size of 1;

[0130] In step 202, multi-scale feature fusion processing is performed on the feature maps of T3, T4 and D5, and then the features of the two polarizations are fused to finally obtain four bilinear vectors. Specifically, multi-scale feature fusion processing is first performed on the feature maps of the transition dimension reduction layer T3, the transition dimension reduction layer T4 and the dense connection layer D5; then the features of the transition dimension reduction layer T3 and the transition dimension reduction layer T4 are fused, and the features of the two radar ship images are fused to obtain a first bilinear vector v ms .

[0131] The feature fusion layer FBC consists of two parallel fully connected layers f1 and f2, followed by a max-pooling layer M. The pooling kernel window size of the max-pooling layer M is 4. Features of the two polarizations located in the deep densely connected layers at the transition dimensionality reduction layers T1, T2, T3, T4 and D5 are extracted respectively. , , , , , , , , , ;

[0132] T3 features in the transitional dimensionality reduction layer of VV polarization The VV polarization features located in the transition dimensionality reduction layers T1, T2, T4, and D5 are analyzed. , , and Samples were obtained through downsampling layer DS2, downsampling layer DS3, and upsampling layer U1, respectively. At the same size, they are then added along the channel dimension and merged using a 1×1 convolution kernel C1 to obtain the feature F of the VV-polarized transition dimensionality reduction layer T3 multi-scale feature fusion. vv_3 The specific formula is as follows:

[0133] F vv_3 =C1(DS2(F) vv_1 )+DS3(F vv_2 )+F vv_3 +U1(F vv_4 )+U1(F vv_5 (3)

[0134] T3 characteristics of VH polarization The VH polarization features located in the transition dimensionality reduction layers T1, T2, T4, and D5 are described above. , , and Samples were obtained through downsampling layer DS2, downsampling layer DS3, and upsampling layer U1, respectively. At the same size, they are then added along the channel dimension and merged using a 1×1 convolution kernel C1 to obtain the feature F of the VH-polarized transition dimensionality reduction layer T3 multi-scale feature fusion. vh_3 The specific formula is as follows:

[0135] Fvh_3 ′= C1(DS2(F vh_1 )+ DS3(F vh_2 )+ F vh_3 + U1(F vh_4 )+ U1(F vh_5 )) (4)

[0136] Similarly, on the feature of the transition dimension reduction layer T4 of the VV polarization, the features , , and located in the transition dimension reduction layer T1, the transition dimension reduction layer T2, the transition dimension reduction layer T4 of the transition dimension reduction layer T3, and the densely connected layer D5 of the VV polarization are respectively sampled to the same size as by the downsampling layer DS1, the downsampling layer DS2 and the downsampling layer DS3, and then added along the channel dimension and merged using the 1×1 convolution kernel C1 to obtain the feature F vv_4 ′ of the transition dimension reduction layer T4 of the VV polarization multi-scale feature fusion.

[0137] F vv_4 ′= C1(DS1(F vv_1 )+ DS2(F vv_2 )+ DS3(F vv_3 )+ F vv_4 + F vv_5 ) (5)

[0138] On the feature of the transition dimension reduction layer T4 of the VH polarization, the features , , and located in the transition dimension reduction layer T1, the transition dimension reduction layer T2, the transition dimension reduction layer T3, and the densely connected layer D5 of the VH polarization are respectively sampled to the same size as by the downsampling layer DS1, the downsampling layer DS2 and the downsampling layer DS3, and then added along the channel dimension and merged using the 1×1 convolution kernel C1 to obtain the feature F vh_4 ′ of the transition dimension reduction layer T4 of the VH polarization multi-scale feature fusion.

[0139] F vh_4 ′= C1(DS1(F vh_1 )+ DS2(F vh_2 )+ DS3(F vh_3 )+ F vh_4 + F ch_5 ) (6)

[0140] In the densely connected layer D5 feature The VV polarization features located in the transition dimensionality reduction layers T1, T2, T3, and dense connection layer T4 are described above. , , and Samples were obtained through downsampling layers DS1, DS2, and DS3 respectively. At the same size, they are then added along the channel dimension and merged using a 1×1 convolution kernel C1 to obtain the VV-polarized densely connected layer D5, which fuses multi-scale features F. vv_5 The specific formula is as follows:

[0141] F vv_5 =C1(DS1(F) vv_1 )+DS2(F vv_2 )+DS3(F vv_3 )+F vv_4 +F vv_5 (7)

[0142] Features of D5 densely connected layer in VH polarization The VH polarization features located in the transition dimensionality reduction layers T1, T2, T3, and dense connection layer T4 are described above. , , and Samples were obtained through downsampling layers DS1, DS2, and DS3 respectively. At the same size, they are then added along the channel dimension and merged using a 1×1 convolution kernel C1 to obtain the feature F of the VH-polarized densely connected layer D5, which is a multi-scale feature fusion layer. vh_5 The specific formula is as follows:

[0143] F vh_5 =C1(DS1(F) vh_1 )+DS2(F vh_2 )+DS3(F vh_3 )+F vh_4 +F vh_5 (8)

[0144] In VV polarization, the feature F of the transition dimensionality reduction layer T3 and the transition dimensionality reduction layer T4 after multi-scale feature fusion. vv_3 ′ and F vv_4 ′Up, F vv_3 'Sampling is performed at F through the downsampling layer DS3.' vv_4 At the same size, they are then added along the channel dimension and merged using a 1×1 convolution kernel C1 to obtain the VV polarization global multi-scale features C. vv_ms The specific formula is as follows:

[0145] C vv_ms =C1(DS3(F) vv_3 ′)+F vv_4 ′) (9)

[0146] Similarly, regarding VH polarization, the feature F of the transition dimensionality reduction layer T3 and the transition dimensionality reduction layer T4 after multi-scale feature fusion... vh_3 ′ and F vh_4 ′Up, F vh_3 'Sampling is performed at F through the downsampling layer DS3.' vh_4 At the same size, they are then added along the channel dimension and merged using a 1×1 convolution kernel C1 to obtain the VH polarization global multi-scale features C. vh_ms The specific formula is as follows:

[0147] C vh_ms =C1(DS3(F) vh_3 ′)+F vh_4 ′) (10)

[0148] Finally, a feature fusion layer (FBC) is used to feed the result of fusing the two polarization multi-scale methods into two parallel fully connected layers (f1 and f2). The result from the fully connected layers is then multiplied by a dot product and fed into a max pooling layer (M) to obtain the first bilinear vector v. ms The step of obtaining the first bilinear vector v ms As shown in the following formula:

[0149] v ms =M(f1(C vv_ms ′)*f2(C vh_ms ′)) (11)

[0150] Among them, C vv_ms ′ and C vh_ms ′ represents the global multi-scale features of VV polarization and VH polarization, f1 and f2 represent two parallel fully connected layers, * represents the dot product of two vectors, and M represents the max pooling layer.

[0151] In one embodiment, the method for constructing the layers of the cross-polarity attention layer in step 203 is as follows:

[0152] The first-order attention layer F SE It is composed of a global average pooling layer, a fully connected layer, a batch normalization (BN) layer, a ReLU layer, a fully connected layer, and a sigmoid layer connected in series.

[0153] The second-order attention layer S SE It consists of a grouped bilinear pooling layer, a fully connected layer, a BN layer, a ReLU layer, a fully connected layer, and a Sigmoid layer;

[0154] The two polarizations obtained in step 202 are located at D5, and the characteristic F is... vh_5 ′ and F vv_5 ′, respectively passing through the first-order attention layer F SE and second-order attention layer S SE This yields the first-order attention vector v. vh and the second-order attention vector v vv The first-order attention vector v vh With F vv_5 Multiply by ', and multiply the second-order attention vector v vv With F vh_5 Multiply by ' to obtain the feature C after multiplication. vh ′、C vv The specific formula is as follows, which constitutes the transpolar attention layer;

[0155] C vh ′=F vh_5 ′×v vv (12)

[0156] C vv ′=F vv_5 ′×v vh (13)

[0157] Then multiply the feature C vh ′、C vv The vector is fed into the feature fusion layer FBC to obtain the second bilinear vector v. vh-vv for:

[0158] v vh-vv =M(f1(C vh ′)*f2(C vv ′));

[0159] Simultaneously, the feature F obtained by fusing the two polarizations through multi-scale features vv ′ and F vh Performing single-polarity feature fusion (FBC) operations on each vector yields a third bilinear vector v. vh-vh The fourth bilinear vector v vv-vv The specific formula is shown below:

[0160] v vh-vh =M(f1(C vh ′)*f2(C vh ′)) (14)

[0161] v vv-vv =M(f1(C vv ′)*f2(C vv ′)) (15).

[0162] In one embodiment, the steps for constructing the adaptive learning loss weight layer in step 204 are as follows:

[0163] Step 2041, the first bilinear vector v ms The second bilinear vector v vh-vv The third bilinear vector v vh-vh and the fourth bilinear vector v vv-vv The data is fed into a Softmax classifier layer, and the cross-entropy of each loss term is calculated. The results are as follows: Cross-entropy loss L of the first bilinear vector. Mutil_scale The second bilinear SAR ship target classification method based on multi-scale transpolar attention networks uses the cross-entropy loss L vector. VH-VV The cross-entropy loss L of the third bilinear vector VH-VH The cross-entropy loss L of the fourth bilinear vector VV-VV ;

[0164] Step 2042 uses the gradient norm from the adaptive learning weight method to construct the loss function L. grad The weights corresponding to each loss term are used to learn the weights, and the corresponding gradient l2 norm is calculated based on each loss term. Calculate the mean of the gradient norm for each loss term. Using the 0th iteration as a baseline, calculate the inverse training rate for each loss term at the current iteration. Finally, based on each loss Calculate its corresponding relative anti-training rate r i (t) represents the speed at which each loss term is trained compared to all losses; the specific calculation formula is shown below:

[0165] The loss function used to update the weight values ​​of each loss term:

[0166] (16)

[0167] Among them, L grad It is the l1 norm of the difference between the gradient norm of the i-th loss and the mean of the gradient norms of all terms multiplied by the relative anti-training rate of the i-th loss, where i is the number of each loss term, i∈{1,2,3,4}.

[0168] Calculate the gradient norm of each loss term: (17)

[0169] in, It represents the partial derivative of the i-th loss multiplied by its weight with respect to the parameter W; The L2 norm of the gradient of the i-th loss multiplied by the i-th weight with respect to the parameter w;

[0170] Calculate the mean of the gradient norm: (18)

[0171] in, To calculate the mean of the gradient norm for each loss term;

[0172] Calculate the inverse training rate for each loss term in the current iteration: (19)

[0173] Among them, L i (0) represents the loss of the i-th term at the 0th iteration;

[0174] Calculate the relative anti-training rate for each loss: (20)

[0175] Where α is a hyperparameter, α∈[0,2), is a balance coefficient, and r i (t) is the inverse training rate for each loss term at the current iteration. Divide by the mean of the inverse training rate at the current iteration of all terms loss. .

[0176] In one embodiment, step 3 uses the already constructed loss function L. ALL A multi-scale, cross-polarity attention network is trained until the loss function L is reached. ALL Convergence yields the trained multi-scale cross-polarity attention network Ω′, specifically referring to the overall loss function L. ALL :

[0177] L ALL =w1L Mutil_scale +w2L VH-VV +w3L VH-VH +w4L VV-VV (1)

[0178] The multi-scale cross-polarity attention network is trained as follows: the training process is based on the overall loss function L. ALL The value of is used for backpropagation, and the gradient is obtained according to the chain rule. The gradient is then used to update all parameters in the network. Simultaneously, the loss function L is used based on the weight values ​​of each loss term. grad Similarly, backpropagation is performed to update each weight wi of the loss, i∈{1,2,3,4};

[0179] The expanded training set Φ is randomly divided into each x After training the network Ω at least three times to obtain the trained network Ω′, test the network Φ. c ; Conduct tests and record the median value of each indicator.

[0180] In one embodiment, step 4 will use the test set Φc The data is input into a pre-trained multi-scale transpolarity attention network Ω′ to obtain the predicted classification results of radar ship images. The predicted classification results are then compared with the test set Φ. c The actual categories are compared, and the final target evaluation index is obtained according to the calculation method of the evaluation index. The final target evaluation index is then compared with the evaluation indexes of other SAR ship classification methods. Specifically, the predicted classification result is the probability that each sample is predicted to be in each category, and the category with the highest probability is selected as the predicted category of that sample. Based on the classification result and the test set Φ c The true categories are compared; the comparison results are calculated as overall accuracy OA, precision P, recall R and metric F1, and then compared with methods related to SAR ship classification.

[0181] Wherein, overall accuracy OA is the number of correctly predicted samples / total number of samples, precision P is the number of correctly predicted positive cases / total number of predicted positive cases, recall R is the number of correctly predicted positive cases / total number of actual positive cases, and metric F1 is the harmonic mean of precision P and recall R.

[0182] This invention provides a SAR ship target classification method based on a multi-scale cross-polarity attention network, which solves the problems of limited available information in SAR images, the presence of speckle noise, trailing noise, and insufficient utilization of polarization information. This invention utilizes FBC for fusion, achieving better results compared to grouped bilinear pooling. Furthermore, the use of multi-scale information overcomes the disadvantage of limited SAR ship information, fully utilizing contextual information. Cross-polarity attention is more suitable for SAR ship classification than channel attention and spatial attention, allowing the two polarities to guide each other and fully utilize their complementary information. Adaptive adjustment of loss weights makes network training more balanced, unaffected by any single loss factor, improving the effectiveness of the network framework in extracting SAR ship target features and improving classification performance.

[0183] Example 1

[0184] This invention provides a SAR ship target classification method based on a multi-scale transpolar attention network. The SAR ship target classification method based on a multi-scale transpolar attention network consists of the following steps (see...). Figure 1 ):

[0185] 1. Obtain the training set Φ x and test set Φ c Two types of radar ship images were selected from the OpenSARShip dataset: VV polarization (vertical transmit, vertical receive) and VH polarization (vertical transmit, horizontal receive). These images were then divided into a training set Φ at an 8:2 ratio. xand test set Φ c ;

[0186] 2. For the training set Φ x Expanding to obtain Φ x ′

[0187] The training set Φ was modified using flipping, rotating, translating, and noise-adding methods. x The training set Φ is augmented by a factor of 8. x ′;

[0188] 3. Construct a multi-scale, transpolar attention network Ω;

[0189] Reference Figure 2 The multi-scale cross-polarity attention network consists of a deep dense connection layer, a multi-scale feature fusion layer, a cross-polarity attention layer, and an adaptive learning weight layer connected in series.

[0190] (3a) Construct a deep, densely connected layer:

[0191] The deep dense connection layer consists of two identical branches, which simultaneously input radar images of two polarizations. Each branch is composed of the base layer S and the dense connection layer D1, transition dimensionality reduction layer T1, dense connection layer D2, transition dimensionality reduction layer T2, dense connection layer D3, transition dimensionality reduction layer T3, dense connection layer D4, transition dimensionality reduction layer T4, and dense connection layer D5 connected in series.

[0192] The base layer S consists of two identical convolutional blocks connected in series. Each convolutional block consists of a convolutional layer L1, a batch normalization layer L2, and an activation function layer L3 connected in series. The activation function layer L3 outputs a nonlinear mapping ReLU(x) as follows: ReLU(x) = max(0,x) where x is the output of the batch normalization layer L2. The kernel size of the convolutional layers is 3, and the stride is 1.

[0193] The densely connected layer D1, with a growth rate of 3, consists of three identical convolutional blocks connected in series, each convolutional block being a batch normalization layer. Activation function layer Convolutional layer dropout layer They are connected in series, with each convolutional kernel size of 3 and a stride of 1. n represents the nth convolutional block, where n∈{1,2,3}.

[0194] Transitional dimensionality reduction layer T1 is derived from batch normalization layer T1 1 Activation function layer Convolutional layer T4 i Average pooling layer The convolutional kernels are connected in series, each with a kernel size of 3 and a stride of 1. The downsampling kernel window size is 2 and the stride is 2.

[0195] D2, a densely connected layer with a growth rate of 6, consists of three identical convolutional blocks cascaded together, each block consisting of a batch normalization layer. Activation function layer Convolutional layer dropout layer They are connected in series, with a kernel size of 3 and a stride of 1;

[0196] The transition dimensionality reduction layer T2 is composed of the batch normalization layer T1. 2 Activation function layer Convolutional layer Average pooling layer The convolutional kernels are connected in series, each with a kernel size of 3 and a stride of 1. The downsampling kernel window size is 2 and the stride is 2.

[0197] The densely connected layer D3, with a growth rate of 9, consists of three identical convolutional blocks connected in series, each convolutional block being a batch normalization layer. Activation function layer Convolutional layer dropout layer They are connected in series, with a kernel size of 3 and a stride of 1;

[0198] The transition dimensionality reduction layer T3 consists of the batch normalization layer T1. 3 Activation function layer Convolutional layer Average pooling layer The convolutional kernels are connected in series, each with a kernel size of 3 and a stride of 1. The downsampling kernel window size is 2 and the stride is 2.

[0199] The densely connected layer D4, with a growth rate of 12, consists of three identical convolutional blocks connected in series, each convolutional block being a batch normalization layer. Activation function layer Convolutional layer dropout layer They are connected in series, with a kernel size of 3 and a stride of 1;

[0200] The aforementioned transitional dimensionality reduction layer T4 is composed of the batch normalization layer T1. 4 Activation function layer Convolutional layer Average pooling layer The convolutional kernels are connected in series, each with a kernel size of 3 and a stride of 1. The downsampling kernel window size is 2 and the stride is 2.

[0201] The densely connected layer D5, with a growth rate of 15, consists of three identical convolutional blocks connected in series, each convolutional block being a batch normalization layer. Activation function layer Convolutional layer dropout layer They are connected in series, with a kernel size of 3 and a stride of 1;

[0202] (3b) Constructing a multi-scale feature fusion layer:

[0203] This layer consists of a downsampling layer DS1, a downsampling layer DS2, a downsampling layer DS3, an upsampling layer U1, a feature fusion layer FBC, and a 1×1 convolutional kernel C1.

[0204] The methods for constructing each layer of the multi-scale feature fusion layer are as follows:

[0205] The downsampling layer DS1 has a downsampling kernel window size and stride set according to the size of the feature map. In this embodiment, the downsampling kernel window size is set to 8 and the stride is set to 8.

[0206] The downsampling layer DS2 has a downsampling kernel window size and stride set according to the size of the feature map. In this embodiment, the downsampling kernel window size is set to 4 and the stride is set to 4.

[0207] The downsampling layer DS3 has a downsampling kernel window size and stride set according to the size of the feature map. In this embodiment, the downsampling kernel window size is set to 2 and the stride is set to 2.

[0208] The upsampling layer U1 has an upsampling kernel window size and stride set according to the size of the feature map. In this embodiment, the upsampling kernel window size is set to 2 and the stride is set to 2.

[0209] The feature fusion layer FBC, as referred to Figure 5 It consists of two parallel fully connected layers f1 and f2, and then a Max-Pooling layer M connected in series. The size of the pooling kernel window of the Max-Pooling layer M is set according to the size of the feature map. In this embodiment, it is set to 4.

[0210] The two polarizations are characterized at T1, T2, T3, T4, and D5 in the deep densely connected layers. , , , , , , , , , Extract them separately.

[0211] Reference Figure 3 Features of the T3 layer in the transition dimensionality reduction layer of VV polarization The VV polarization features located in the transition dimensionality reduction layers T1, T2, T4, and D5 are described above. , , and Samples were obtained through downsampling layer DS2, downsampling layer DS3, and upsampling layer U1, respectively. At the same size, they are then added along the channel dimension and merged using a 1×1 convolution kernel C1 to obtain the multi-scale feature fusion feature F. vv_3 The specific formula is as follows:

[0212] F vv_3 =C1(DS2(F) vv_1 )+DS3(F vv_2 )+F vv_3 +U1(F vv_4 )+U1(F vv_5 ))

[0213] Reference Figure 3 T3 characteristics of VH polarization The VH polarization features located in the transition dimensionality reduction layers T1, T2, T4, and D5 are described above. , , and Samples were obtained through downsampling layer DS2, downsampling layer DS3, and upsampling layer U1, respectively. At the same size, they are then added along the channel dimension and merged using a 1×1 convolution kernel C1 to obtain the multi-scale feature fusion feature F. vh_3 The specific formula is as follows:

[0214] F vh_3 =C1(DS2(F) vh_1 )+DS3(F vh_2 )+F vh_3 +U1(F vh_4 )+U1(F vh_5 ))

[0215] Similarly, in the T4 characteristic of VV polarization The VV polarization features located in the transition dimensionality reduction layers T1, T2, T3, and D5 are described above. , , and Samples were obtained through downsampling layers DS1, DS2, and DS3 respectively. At the same size, they are then added along the channel dimension and merged using a 1×1 convolution kernel C1 to obtain the multi-scale feature fusion feature F. vv_4 The specific formula is as follows:

[0216] F vv_4 =C1(DS1(F) vv_1 )+DS2(F vv_2 )+DS3(F vv_3 )+F vv_4 +F vv_5 )

[0217] T4 characteristics of VH polarization The VH polarization features located in the transition dimensionality reduction layers T1, T2, T3, and D5 are described above. , , and Samples were obtained through downsampling layers DS1, DS2, and DS3 respectively. At the same size, they are then added along the channel dimension and merged using a 1×1 convolution kernel C1 to obtain the multi-scale feature fusion feature F. vh_4 The specific formula is as follows:

[0218] F vh_4 =C1(DS1(F) vh_1 )+DS2(F vh_2 )+DS3(F vh_3 )+F vh_4 +F vh_5 )

[0219] D5 characteristics of VV polarization The VV polarization features located in the transition dimensionality reduction layers T1, T2, T3, and dense connection layer T4 are described above. , , and Samples were obtained through downsampling layers DS1, DS2, and DS3 respectively. At the same size, they are then added along the channel dimension and merged using a 1×1 convolution kernel C1 to obtain the multi-scale feature fusion feature F. vv_5 The specific formula is as follows:

[0220] F vv_5 =C1(DS1(F) vv_1 )+DS2(F vv_2 )+DS3(F vv_3 )+F vv_4 +F vv_5 )

[0221] D5 characteristics of VH polarization The VH polarization features located in the transition dimensionality reduction layers T1, T2, T3, and dense connection layer T4 are described above. , , and Samples were obtained through downsampling layers DS1, DS2, and DS3 respectively. At the same size, they are then added along the channel dimension and merged using a 1×1 convolution kernel C1 to obtain the multi-scale feature fusion feature F. vh_5 The specific formula is as follows:

[0222] F vh_5 =C1(DS1(F) vh_1 )+DS2(F vh_2 )+DS3(F vh_3 )+F vh_4 +F vh_5 )

[0223] In VV polarization, the feature F of the transition dimensionality reduction layer T3 and the transition dimensionality reduction layer T4 after multi-scale feature fusion. vv_3 'and

[0224] F vv_4 ′Up, F vv_3 'Sampling is performed at F through the downsampling layer DS3.' vv_4 At the same size, they are then added along the channel dimension and merged using a 1×1 convolution kernel C1 to obtain the VV polarization global multi-scale features C. vv_ms The specific formula is as follows:

[0225] C vv_ms =C1(DS3(F) vv_3 ′)+F vv_4 ′)

[0226] Similarly, regarding VH polarization, the feature F of the transition dimensionality reduction layer T3 and the transition dimensionality reduction layer T4 after multi-scale feature fusion... vh_3 ′ and F vh_4 ′Up, F vh_3 'Sampling is performed at F through the downsampling layer DS3.' vh_4 At the same size, they are then added along the channel dimension and merged using a 1×1 convolution kernel C1 to obtain the VH polarization global multi-scale features C. vh_ms The specific formula is as follows:

[0227] C vh_ms =C1(DS3(F) vh_3 ′)+F vh_4 ′)

[0228] Finally, a feature fusion layer (FBC) is used to feed the result of fusing the two polarization multi-scale methods into two parallel fully connected layers (f1 and f2). The result from the fully connected layers is then multiplied by a dot product and fed into a max pooling layer (M) to obtain the first bilinear vector v. ms The step of obtaining the first bilinear vector v ms As shown in the following formula:

[0229] v ms =M(f1(C vv_ms ′)*f2(C vh_ms ′))

[0230] Among them, C vv_ms ′ and C vh_ms ′ represents the global multi-scale features of VV polarization and VH polarization, f1 and f2 represent two parallel fully connected layers, * represents the dot product of two vectors, and M represents the max pooling layer.

[0231] (3c) Constructing a cross-polarity attention layer:

[0232] Reference Figure 4 This layer consists of a first-order attention layer F. SE Second-order attention layer S SE Composition, the first-order attention layer F SE It consists of a global average pooling layer, a fully connected layer, a batch normalization (BN) layer, a ReLU layer, a fully connected layer, a batch normalization (BN) layer, and a sigmoid layer connected in series.

[0233] The second-order attention layer S SE It consists of a grouped bilinear pooling layer, a fully connected layer, a BN layer, a ReLU layer, a fully connected layer, a BN layer, and a Sigmoid layer;

[0234] The two polarizations obtained in step (3b) are located at D5, F vv_5 ′ and F vh_5 ′, respectively passing through the first-order attention layer F SE and second-order attention layer S SE This yields the first-order attention vector v. vh and the second-order attention vector v vv The first-order attention vector v vh With F vv_5 Multiply by ', and multiply the second-order attention vector v vv With F vh_5 Multiply by ' to obtain the feature C after multiplication. vh ′、C vv The specific formula is as follows, which constitutes the transpolar attention layer;

[0235] C vh′=F vh_5 ×v vv

[0236] C vv ′=F vv_ 5′×v vh

[0237] Then multiply the feature C vh ′、C vv The vector is fed into the feature fusion layer FBC to obtain the second bilinear vector v. vh-vv The specific formula is shown below:

[0238] v vh-vv =M(f1(C vh ′)*f2(C vv ′));

[0239] Simultaneously, the feature F obtained by fusing the two polarizations through multi-scale features vv ′ and F vh Performing single-polarity feature fusion (FBC) operations on each vector yields a third bilinear vector v. vh-vh The fourth bilinear vector v vv-vv The specific formula is shown below:

[0240] v vh-vh =M(f1(C vh ′)*f2(C vh ′));

[0241] v vv-vv =M(f1(C vv ′)*f2(C vv ′));

[0242] (3d) Constructing an adaptive learning loss weight layer:

[0243] Reference Figure 2 This layer consists of a parameter-shared fully connected layer (FC), a Softmax classifier, and a GradNorm layer concatenated. In step (3b), the first bilinear vector v is obtained. ms And in step (3c), the second bilinear vector v is obtained. vh-vv The third bilinear vector v vh-vh and the fourth bilinear vector v vv-vv Four bilinear vectors are fed into the FC layer and the Softmax classifier layer. The cross-entropy of each loss term is calculated, and the results are as follows: L Mutil_scale L VH-VV L VH-VH L VV-VV The gradient norm from the adaptive learning weight method is used to construct the loss function L.grad The weights corresponding to each loss term are used to learn the weights, and the corresponding gradient l2 norm is calculated based on each loss term. Calculate the mean of the gradient norm for each loss term. Using the 0th iteration as a baseline, calculate the inverse training rate for each loss term at the current iteration. Finally, based on each loss Calculate its corresponding relative anti-training rate r i (t) represents the speed at which each loss term is trained compared to all losses; the specific calculation formula is shown below:

[0244] The loss function used to update the weight values ​​of each loss term:

[0245] Among them, L grad It is the l1 norm of the difference between the gradient norm of the i-th loss and the mean of the gradient norms of all terms multiplied by the relative anti-training rate of the i-th loss, where i is the number of each loss term, i∈{1,2,3,4}.

[0246] Calculate the gradient norm of each loss term:

[0247] in, It represents the partial derivative of the i-th loss multiplied by its weight with respect to the parameter W; The L2 norm of the gradient of the i-th loss multiplied by the i-th weight with respect to the parameter w;

[0248] Calculate the mean of the gradient norm:

[0249] in, To calculate the mean of the gradient norm for each loss term;

[0250] Calculate the inverse training rate for each loss term in the current iteration:

[0251] Among them, L i (0) represents the loss of the i-th term at the 0th iteration;

[0252] Calculate the relative anti-training rate for each loss:

[0253] Where α is a hyperparameter, α∈[0,2), is a balance coefficient, and r i (t) is the inverse training rate for each loss term at the current iteration. Divide by the mean of the inverse training rate at the current iteration of all terms loss. In this embodiment, α = 1 is selected.

[0254] 4. Training a multi-scale transpolar attention network Ω

[0255] The expanded training set Φ x The input is fed into the constructed multi-scale transpolar attention network Ω, and the output is the classification result, which is then processed using the proposed objective loss function L. ALL A multi-scale, cross-polarity attention network was trained. Compared to the overall accuracy of other works (as seen in the experimental details), this embodiment sets the overall accuracy on the test set to 89%, and the loss function L... ALL If convergence is achieved, training is complete, resulting in a well-trained multi-scale cross-polarity attention network Ω′. Specifically, this refers to using the proposed target loss function L. ALL :

[0256] L ALL =w1L Mutil_scale +w2L VH-VH +w3L VV-VV +w4L VH-VV The multi-scale cross-polarity attention network is trained as follows: The training process is based on the target loss function L... ALL The value of is used for backpropagation, and the gradient is obtained according to the chain rule. The gradient is then used to update all parameters in the network. Simultaneously, the loss function L is used based on the weight values ​​of each loss term. grad Similarly, perform backpropagation to update each weight wi of the loss, i∈{1,2,3,4}; and expand each randomly partitioned training set Φ. x After training the network Ω at least three times to obtain the trained network Ω′, test the network Φ. c ; Conduct tests and record the median value of each indicator.

[0257] 5. Test the multi-scale transpolar attention network to obtain ship classification results.

[0258] The ship test set Φ c The data is input into a pre-trained multi-scale transpolarity attention network Ω′ to obtain the classification results of radar ship images. The predicted classification results are then compared with the test set Φ. c The actual categories are compared, and the final target evaluation index is obtained according to the calculation method of the evaluation index. The final target evaluation index is then compared with the evaluation index of other SAR ship classification methods. See Table 2 for details. Where OA is the number of correctly predicted samples / total number of samples, P is the number of correctly predicted positive examples / total predicted positive examples, R is the number of correctly predicted positive examples / total actual positive examples, and F1 is the harmonic mean of P and R.

[0259] The effectiveness of this invention can be further illustrated by the following experimental data:

[0260] I. Experimental Conditions

[0261] 1) Experimental data

[0262] This experiment uses the reorganized OpenSARShip dataset. Since the number of ships in each class of the OpenSARShip dataset varies greatly and some data contains two ship targets in one image, which will affect the classification accuracy, the dataset was reorganized. The number of ships in each class of the reorganized data is shown in Table 1.

[0263] Table 1: Number of various types of ships after compilation

[0264] Class Tanker Container ship Bulk carrier Cargo ship General cargo ship Training 280 167 632 707 91 Test 80 50 170 270 30 Total 360 217 802 977 121 Three category expansion 1336 1336 1336 Five category expansion 728 728 728 728 728

[0265] 2) Evaluation Criteria

[0266] To evaluate the effectiveness of the proposed method, we selected the most commonly used metrics, such as overall accuracy (OA), defined as the ratio of the number of correctly classified ship samples to the total number of samples in the test set. In addition, precision (P), recall (R), and the F1 score were used for comprehensive evaluation. We used a randomly partitioned augmented training set Φ for each... x After training the network Ω three times to obtain the trained network Ω′, the test set Φ is then used. c Conduct tests and record the median value for each metric.

[0267] II. Experiment Content

[0268] The experimental data described above were compared using the method of this invention and the latest method on three-class datasets and five-class datasets, respectively. The performance parameter comparison results are shown in Tables 2 and 3.

[0269] Table 2 compares the performance parameters of the latest methods on three test sets (comparing methods related to SAR ship classification).

[0270]

[0271] Table 3 Overall accuracy and standard deviation of ablation studies using three test sets

[0272]

[0273] As can be seen from Table 2, compared with existing methods, the present invention achieves better results on each metric. This is because the network designed in the present invention can extract the features of the samples better, and utilizes multi-scale information to obtain richer contextual information. It also makes the two polarizations more complementary and mutually guide each other to learn their respective advantages, effectively exploring the polarization information in dual-polarized ship images. Combined with these feature information, the present invention exhibits better performance.

[0274] As shown in Table 3, the FBC model achieves better results than the grouped bilinear pooling model, indicating that the FBC model is superior to the grouped bilinear pooling model. Therefore, we subsequently sampled the FBC model. Furthermore, utilizing multi-scale features can obtain richer feature information and fully utilize contextual feature information, which experiments have proven to be effective. Compared to channel attention and spatial attention, cross-polarity attention is more suitable for dual-polarity SAR ship classification, allowing the two polarities to guide each other and fully utilize their complementary information. Adaptive adjustment of loss weights makes network training more balanced, unaffected by any single loss factor, improving the effectiveness of the network framework in extracting SAR ship target features and its classification performance. In summary, this invention adopts the FBC model, utilizes multi-scale features and cross-polarity attention, and also uses adaptive adjustment of loss weights to achieve better network performance.

[0275] The above description is merely a specific example of the present invention and does not constitute any limitation on the present invention. Obviously, those skilled in the art, after understanding the content and principles of the present invention, may make various modifications and changes in form and details without departing from the principles and structure of the present invention. However, these modifications and changes based on the ideas of the present invention are still within the scope of protection of the claims of the present invention.

Claims

1. A SAR ship target classification method based on multi-scale transpolar attention network, characterized in that, The method includes the following steps: Step 1: Select two types of radar ship images from the OpenSARShip dataset: VV polarization (vertical transmit / vertical receive signal) and VH polarization (vertical transmit / horizontal receive signal). Divide the VV and VH polarization ship images into a training set Φ at an 8:2 ratio. x and test set Φ c ; train set Φ x The training set Φ was modified using methods such as flipping, rotating, translating, and adding noise. x The training set Φ is augmented by a factor of k to obtain the augmented training set Φ. x ′, where k≥2, and k is a natural number; Step 2: Construct a multi-scale cross-polarity attention network Ω, which is composed of a deep dense connection layer, a multi-scale feature fusion layer, a cross-polarity attention layer, and an adaptive learning weight layer connected in series. The transpolarization attention layer consists of a first-order attention layer F. SE Second-order attention layer S SE composition; The first-order attention layer F SE It is composed of a global average pooling layer, a fully connected layer, a batch normalization (BN) layer, a ReLU layer, a fully connected layer, and a sigmoid layer connected in series. The second-order attention layer S SE It consists of a grouped bilinear pooling layer, a fully connected layer, a BN layer, a ReLU layer, a fully connected layer, and a Sigmoid layer; Step 3 trains a multi-scale cross-polarity attention network Ω, wherein the multi-scale cross-polarity attention network Ω is trained by obtaining a target loss function L that adaptively learns weights. ALL To train the network, the expanded training set Φ x The input is fed into the constructed multi-scale transpolar attention network Ω, and the output is a classification result, which is the probability that each sample is predicted to belong to each category. The category with the highest probability is selected as the predicted category of the sample. The pre-constructed target loss function L is used. ALL A multi-scale, cross-polarity attention network is trained until the target loss function L is reached. ALL Convergence yields the trained multi-scale transpolar attention network Ω′. Step 4: Test set Φ c The input is fed into a pre-trained multi-scale transpolarity attention network Ω′ to obtain the predicted classification results of radar ship images. These predicted classification results are then compared with the test set Φ. c By comparing the actual categories, and based on the calculation method of the evaluation index, the final target evaluation index is obtained. The final target evaluation index is then compared with the evaluation indexes of other SAR ship classification methods.

2. The SAR ship target classification method based on multi-scale transpolar attention network according to claim 1, characterized in that, In step 1, two types of radar ship images, one with vertical transmission and vertical reception signal VV polarization and the other with vertical transmission and horizontal reception signal VH polarization, are selected from the OpenSARShip dataset. These two types of polarization ship images are then divided into a training set Φ at an 8:2 ratio. x and test set Φ c ; Obtain the training set Φ x The specific steps are as follows: Step 101: Locate the ship data from the acquired OpenSARShip data file, and calibrate the target SAR images corresponding to the data of selected ship types: tankers, container ships, and bulk carriers. Step 102: Based on the category and location information of the ship slices in the OpenSARShip dataset, re-crop the calibrated target SAR image into 64×64 ship slices. Step 103 replaces redundant ship slices in the cropped SAR image with background, ensuring that each slice contains only one ship target, thus obtaining the training set Φ. x .

3. The SAR ship target classification method based on a multi-scale transpolar attention network according to claim 1, characterized in that, In step 1, the training set Φ x The training set Φ was modified using methods such as flipping, rotating, translating, and adding noise. x The training set Φ is augmented by a factor of k to obtain the augmented training set Φ. x ′, The specific steps include the following: Step 1031 will use the training set Φ x Perform horizontal and vertical flips respectively to obtain a training set that is doubled in size; Step 1032 will use the training set Φ x Rotate clockwise by 90°, 180°, and 270° respectively to obtain a training set that is 3 times larger; Step 1033 will use the training set Φ x The ship target in the training set is arbitrarily translated by [-5,5] pixels in the four directions of up, down, left, and right to obtain a training set that is doubled in size; Step 1034 adds Gaussian noise with a mean of 0 and a variance of 0.001 to the training set Φ. x This results in a training set that is doubled in size. Step 1035 expands the training set obtained after steps 104 to 107 and the original training set Φ x Together, they form the final training set Φ, which is expanded by 8 times. x ′.

4. The SAR ship target classification method based on a multi-scale transpolar attention network according to claim 1, characterized in that, The multi-scale cross-polarity attention network Ω in step 2 is composed of a deep dense connection layer, a multi-scale feature fusion layer, a cross-polarity attention layer, and an adaptive learning weight layer connected in series; the specific steps are as follows: Step 201 constructs a deep dense connection layer. The deep dense connection layer consists of two identical branches, and two polarized radar images are input simultaneously. Each branch is composed of a base layer S and a dense connection layer D1, a transition dimensionality reduction layer T1, a dense connection layer D2, a transition dimensionality reduction layer T2, a dense connection layer D3, a transition dimensionality reduction layer T3, a dense connection layer D4, a transition dimensionality reduction layer T4, and a dense connection layer D5, which are connected in series. The features of each layer of the deep dense connection layer are extracted. Step 202 constructs a multi-scale feature fusion layer, which consists of m downsampling layers DS. m The system consists of one upsampling layer U1, one feature fusion layer FBC, and one 1×1 convolutional kernel C1. Multi-scale feature fusion is performed on the feature maps of T3, T4, and D5 respectively. Then, the features of the two polarizations are fused to obtain the first bilinear vector v. ms ; Step 203: Construct a cross-polarity attention layer, and perform multi-scale fusion of the two radar ship images on the densely connected layer D5 to obtain the multi-scale fusion result F. vh and multi-scale fusion results F vv The multi-scale fusion result F vh Feed into the first-order attention layer F SE This yields the transpolarization attention vector v. vh The multi-scale fusion result F vv Feed into the second-order attention layer S SE This yields the transpolarization attention vector v. vv The transpolar attention vector v vh Multiply by the multi-scale fusion result F vv Obtain the transpolarization attention feature F vv ′, will the transpolarization attention vector v vv Multiply by the multi-scale fusion result F vh Obtain the transpolarization attention feature F vh Then, the transpolarization attention feature F vv ′ and transpolarization attention feature F vh The vector is fed into the feature fusion layer FBC to obtain the second bilinear vector v. vh-vv Simultaneously, the transpolarity attention feature F vh The vector is fed into the feature fusion layer FBC to obtain the third bilinear vector v. vh-vh Transpolar attention feature F vv The vector is fed into the feature fusion layer FBC to obtain the fourth bilinear vector v. vv-vv ; Step 204: Construct an adaptive learning loss weight layer, wherein the adaptive learning loss weight layer will convert the first bilinear vector v ms The second bilinear vector v vh-vv The third bilinear vector v vh-vh The fourth bilinear vector v vv-vv The cross-entropy loss L of the first bilinear vector obtained after passing through the Softmax classifier layer. Mutil_scale The cross-entropy loss L of the second bilinear vector VH-VH The cross-entropy loss L of the third bilinear vector VV-VV The cross-entropy loss L of the fourth bilinear vector VH-VV To obtain a new loss function L for adaptive learning weights. ALL To train the network and determine the loss function L ALL for: L ALL =w1L Mutil_scale +w2L VH-VH +w3L VV-VV +w4L VH-VV (1) In the formula, L Mutil_scale L represents the cross-entropy loss of the multi-scale feature fusion term. VH-VH L represents the cross-entropy loss of a vertically transmitted, horizontally received VH-polarized radar image. VV-VV L represents the cross-entropy loss of a vertically transmitted, vertically received VV-polarized radar image. VH-VV denoted as , representing the cross-entropy loss of vertically transmitted horizontally received VH-polarized radar images and vertically transmitted vertically received VV-polarized radar images, where w1, w2, w3, and w4 represent the weights of each loss term.

5. The SAR ship target classification method based on a multi-scale transpolar attention network according to claim 4, characterized in that, The base layer S in step 201 consists of two identical convolutional blocks connected in series. Each convolutional block consists of a convolutional layer L1, a batch normalization layer L2, and an activation function layer L3 connected in series. The activation function layer L3 outputs the nonlinear mapping ReLU(x) as follows: ReLU(x) = max(0,x) (2) In equation (2), x is the output of the batch normalized L2 layer, the kernel size is 3, and the stride is 1. The densely connected layers D1, D2, D3, D4, and D5 are composed of three identical convolutional blocks connected in series, each with a batch normalization layer, according to their respective growth rates. Activation function layer Convolutional layer dropout layer These are sequentially connected to form the corresponding densely connected layer convolutional kernel size and stride; where i∈{1,2,3,4,5}, n∈{1,2,3}, and n represents the nth convolutional block; The transition dimensionality reduction layers T1, T2, T3, and T4 are composed of batch normalization layers. Activation function layer Convolutional layer Average pooling layer The convolution kernel size and stride size of the transition dimensionality reduction layer are obtained by sequentially connecting them. The stride size of the corresponding transition dimensionality reduction layer is obtained according to the downsampling kernel window size of the transition dimensionality reduction layer, where i∈[1,2,3,4].

6. The SAR ship target classification method based on a multi-scale transpolar attention network according to claim 4, characterized in that, The multi-scale feature fusion layer in step 202 consists of m downsampling layers DS. m It consists of one upsampling layer U1, one feature fusion layer FBC, and one 1×1 convolutional layer C1; where m≥1, and m is a natural number; The downsampling layer DS m The size and stride of the downsampling kernel window are set according to the size of the feature map; The size and stride of the upsampling kernel window of the upsampling layer U1 are set according to the size of the feature map; The feature fusion layer FBC consists of two parallel fully connected layers f1 and f2, followed by a max-pooling layer M. The size of the pooling kernel window of the max-pooling layer M is set according to the size of the feature map. The convolutional layer C1 has a kernel size of 1×1 and a stride of 1. In step 202, multi-scale feature fusion processing is performed on the feature maps of T3, T4, and D5 respectively, and then the features of the two polarizations are fused to obtain four bilinear vectors. Specifically, multi-scale feature fusion processing is first performed on the feature maps of the transition dimensionality reduction layer T3, the transition dimensionality reduction layer T4, and the dense connection layer D5; then the features of the transition dimensionality reduction layer T3 and the transition dimensionality reduction layer T4 are fused, and then the features of the two radar ship images are fused to obtain the first bilinear vector v. ms ; The feature fusion layer FBC consists of two parallel fully connected layers f1 and f2, followed by a max-pooling layer M. The pooling kernel window size of the max-pooling layer M is 4. The features of the two polarizations located in the deep densely connected layers at the transition dimensionality reduction layers T1, T2, T3, T4 and D5 are extracted respectively. , , , , , , , , , ; T3 features in the transitional dimensionality reduction layer of VV polarization The VV polarization features located in the transition dimensionality reduction layers T1, T2, T4, and D5 are described above. , , and Samples were obtained through downsampling layer DS2, downsampling layer DS3, and upsampling layer U1, respectively. At the same size, they are then added along the channel dimension and merged using a 1×1 convolution kernel C1 to obtain the feature F of the VV-polarized transition dimensionality reduction layer T3 multi-scale feature fusion. vv_3 The specific formula is as follows: F vv_3 ′=C1(DS2(F vv_1 )+DS3(F vv_2 )+F vv_3 +U1(F vv_4 )+U1(F vv_5 )) (3) T3 characteristics of VH polarization The VH polarization features located in the transition dimensionality reduction layers T1, T2, T4, and D5 are described above. , , and Samples were obtained through downsampling layer DS2, downsampling layer DS3, and upsampling layer U1, respectively. At the same size, they are then added along the channel dimension and merged using a 1×1 convolution kernel C1 to obtain the feature F of the VH-polarized transition dimensionality reduction layer T3 multi-scale feature fusion. vh_3 The specific formula is as follows: F vh_3 ′=C1(DS2(F vh_1 )+DS3(F vh_2 )+F vh_3 +U1(F vh_4 )+U1(F vh_5 ))(4) Similarly, in terms of features The VV polarization is located in the transition dimensionality reduction layers T1, T2, T4, T3, and D5 of the transition dimensionality reduction layer. , , and Samples were obtained through downsampling layers DS1, DS2, and DS3 respectively. At the same size, they are then added along the channel dimension and merged using a 1×1 convolution kernel C1 to obtain the feature F of the VV-polarized transition dimensionality reduction layer T4 multi-scale feature fusion. vv_4 The specific formula is as follows: F vv_4 ′=C1(DS1(F vv_1 )+DS2(F vv_2 )+DS3(F vv_3 )+F vv_4 +F vv_5 ) (5) T4 features in the transitional dimensionality reduction layer of VH polarization The VH polarization features located in the transition dimensionality reduction layers T1, T2, T3, and D5 are described above. , , and Samples were obtained through downsampling layers DS1, DS2, and DS3 respectively. At the same size, they are then added along the channel dimension and merged using a 1×1 convolution kernel C1 to obtain the feature F of the VH-polarized transition dimensionality reduction layer T4 multi-scale feature fusion. vh_4 The specific formula is as follows: F vh_4 ′=C1(DS1(F vh_1 )+DS2(F vh_2 )+DS3(F vh_3 )+F vh_4 +F vh_5 ) (6) Features of D5 densely connected layer with VV polarization The VV polarization features located in the transition dimensionality reduction layers T1, T2, T3, and dense connection layer T4 are described above. , , and Samples were obtained through downsampling layers DS1, DS2, and DS3 respectively. At the same size, they are then added along the channel dimension and merged using a 1×1 convolution kernel C1 to obtain the VV-polarized densely connected layer D5, which fuses multi-scale features F. vv_5 The specific formula is as follows: F vv_5 ′=C1(DS1(F vv_1 )+DS2(F vv_2 )+DS3(F vv_3 )+F vv_4 +F vv_5 ) (7) Features of D5 densely connected layer in VH polarization The VH polarization features located in the transition dimensionality reduction layers T1, T2, T3, and dense connection layer T4 are described above. , , and Samples were obtained through downsampling layers DS1, DS2, and DS3 respectively. At the same size, they are then added along the channel dimension and merged using a 1×1 convolution kernel C1 to obtain the feature F of the VH-polarized densely connected layer D5, which is a multi-scale feature fusion layer. vh_5 The specific formula is as follows: F vh_5 ′=C1(DS1(F vh_1 )+DS2(F vh_2 )+DS3(F vh_3 )+F vh_4 +F vh_5 ) (8) In VV polarization, the feature F of the transition dimensionality reduction layer T3 and the transition dimensionality reduction layer T4 after multi-scale feature fusion. vv_3 ′ and F vv_4 ′Up, F vv_3 'Sampling is performed at F through the downsampling layer DS3.' vv_4 At the same size, they are then added along the channel dimension and merged using a 1×1 convolution kernel C1 to obtain the VV polarization global multi-scale features C. vh_ms The specific formula is as follows: C vv_ms ′=C1(DS3(F vv_3 ′)+F vv_4 ′) (9) Similarly, regarding VH polarization, the feature F of the transition dimensionality reduction layer T3 and the transition dimensionality reduction layer T4 after multi-scale feature fusion... vh_3 ′ and F vh_4 ′Up, F vh_3 'Sampling is performed at F through the downsampling layer DS3.' vh_4 At the same size, they are then added along the channel dimension and merged using a 1×1 convolution kernel C1 to obtain the VH polarization global multi-scale features C. vh_ms The specific formula is as follows: C vh_ms ′=C1(DS3(F vh_3 ′)+F vh_4 ′) (10) Finally, a feature fusion layer (FBC) is used to feed the result of fusing the two polarization multi-scale methods into two parallel fully connected layers (f1 and f2). The result from the fully connected layers is then multiplied by a dot product and fed into a max pooling layer (M) to obtain the first bilinear vector v. ms The step of obtaining the first bilinear vector v ms As shown in the following formula: v ms =M(f1(C vv_ms ′)*f2(C vh_ms ′)) (11) Among them, C vv_ms ′ and C vh_ms ′ represents the global multi-scale features of VV polarization and VH polarization, f1 and f2 represent two parallel fully connected layers, * represents the dot product of two vectors, and M represents the max pooling layer.

7. The SAR ship target classification method based on a multi-scale transpolar attention network according to claim 4, characterized in that, The methods for constructing the transpolar attention layer in step 203 are as follows: The two polarizations obtained in step 202 are located at D5, and the characteristic F is... vh_5 ′ and F vv_5 ′, respectively passing through the first-order attention layer F SE and second-order attention layer S SE This yields the first-order attention vector v. vh and the second-order attention vector v vv The first-order attention vector v vh With F vv_5 Multiply by ', and multiply the second-order attention vector v vv With F vh_5 Multiply by ' to obtain the feature C after multiplication. vh ′、C vv The specific formula is as follows, which constitutes the transpolar attention layer; C vh ′=F vh_5 ′×v vv (12) C vv ′=F vv_5 ′×v vh (13) Then multiply the feature C vh ′、C vv The vector is fed into the feature fusion layer FBC to obtain the second bilinear vector v. vh-vv for: v vh-vv =M(f1(C vh ′)*f2(C vv ′)); Simultaneously, the feature F obtained by fusing the two polarizations through multi-scale features vv ′ and F vh Performing single-polarity feature fusion (FBC) operations on each vector yields a third bilinear vector v. vh-vh The fourth bilinear vector v vv-vv The specific formula is shown below: v vh-vh =M(f1(C vh ′)*f2(C vh ′)) (14) v vv-vv =M(f1(C vv ′)*f2(C vv ′)) (15)。 8. The SAR ship target classification method based on a multi-scale transpolar attention network according to claim 4, characterized in that, The steps for constructing the adaptive learning loss weight layer in step 204 are as follows: Step 2041, the first bilinear vector v ms The second bilinear vector v vh-vv The third bilinear vector v vh-vh and the fourth bilinear vector v vv-vv The data is fed into a Softmax classifier layer, and the cross-entropy of each loss term is calculated. The results are as follows: Cross-entropy loss L of the first bilinear vector. Mutil_scale The second bilinear SAR ship target classification method based on multi-scale transpolar attention networks uses the cross-entropy loss L vector. VH-VV The cross-entropy loss L of the third bilinear vector VH-VH The cross-entropy loss L of the fourth bilinear vector VV-VV ; Step 2042 uses the gradient norm from the adaptive learning weight method to construct the loss function L. grad The weights corresponding to each loss term are used to learn the weights, and the corresponding gradient l2 norm is calculated based on each loss term. Calculate the mean of the gradient norm for each loss term. Using the 0th iteration as a baseline, calculate the inverse training rate for each loss term at the current iteration. Finally, based on each loss Calculate its corresponding relative anti-training rate r i (t) represents the speed at which each loss term is trained compared to all losses; the specific calculation formula is shown below: The loss function used to update the weight values ​​of each loss term: （16） Among them, L grad It is the l1 norm of the difference between the gradient norm of the i-th loss and the mean of the gradient norms of all terms multiplied by the relative anti-training rate of the i-th loss, where i is the number of each loss term, i∈{1,2,3,4}. Calculate the gradient norm of each loss term: (17) in, It represents the partial derivative of the i-th loss multiplied by its weight with respect to the parameter W; The L2 norm of the gradient of the i-th loss multiplied by the i-th weight with respect to the parameter w; Calculate the mean of the gradient norm: (18) in, To calculate the mean of the gradient norm for each loss term; Calculate the inverse training rate for each loss term in the current iteration: (19) Among them, L i (0) represents the loss of the i-th term at the 0th iteration; Calculate the relative anti-training rate for each loss: (20) Where α is a hyperparameter, α∈[0,2), is a balance coefficient, and r i (t) is the inverse training rate for each loss term at the current iteration. Divide by the mean of the inverse training rate at the current iteration of all terms loss. .

9. The SAR ship target classification method based on a multi-scale transpolar attention network according to claim 1, characterized in that, In step 3, the already constructed loss function L is used. ALL A multi-scale, cross-polarity attention network is trained until the loss function L is reached. ALL Convergence yields the trained multi-scale cross-polarity attention network Ω′, specifically referring to the overall loss function L. ALL : L ALL =w1L Mutil_scale +w2L VH-VV +w3L VH-VH +w4L VV-VV (1) The multi-scale cross-polarity attention network is trained as follows: the training process is based on the overall loss function L. ALL The value of is used for backpropagation, and the gradient is obtained according to the chain rule. The gradient is then used to update all parameters in the network. Simultaneously, the loss function L is used based on the weight values ​​of each loss term. grad Similarly, backpropagation is performed to update each weight wi of the loss, i∈{1,2,3,4}; The expanded training set Φ is randomly divided into each x After training the network Ω at least three times to obtain the trained network Ω′, test the network Φ. c ; Conduct tests and record the median value of each indicator.

10. The SAR ship target classification method based on a multi-scale transpolar attention network according to claim 1, characterized in that, In step 4, the test set Φ c The data is input into a pre-trained multi-scale transpolarity attention network Ω′ to obtain the predicted classification results of radar ship images. The predicted classification results are then compared with the test set Φ. c The actual categories are compared, and the final target evaluation index is obtained according to the calculation method of the evaluation index. The final target evaluation index is then compared with the evaluation indexes of other SAR ship classification methods. Specifically, the predicted classification result is the probability that each sample is predicted to be in each category, and the category with the highest probability is selected as the predicted category of that sample. Based on the classification result and the test set Φ c Compare with the actual categories; The results of the comparison were calculated as overall accuracy OA, precision P, recall R, and metric F1, and then compared with methods related to SAR ship classification. Wherein, overall accuracy OA is the number of correctly predicted samples / total number of samples, precision P is the number of correctly predicted positive cases / total number of predicted positive cases, recall R is the number of correctly predicted positive cases / total number of actual positive cases, and metric F1 is the harmonic mean of precision P and recall R.

Images